“Heap, Heap, Array!”Getting AroundLet’s Build a Simple Journey Planner Using SQL
Let’s build a simple journey planner using SQL (part 2 of probably more than 2)
Yet another blog post where I create an unmaintainable mess using our favourite yet very inadequate programming language, SQL.
Last month we started our humble journey on a path towards becoming an ass-hat developer who builds journey planners using SQL. In the previous blog I described the data model for our journey planner and introduced a simple yet ludicrously inefficient pathfinding algorithm in SQL that generates a list of possible paths from one station to another.
Today, we’re continuing the idiocy by writing SQL code that generates travel plans so that you know where you need to go and when.
Recap
Let’s start with a short recap of some of the things we discussed last time. Feel free to skip this section if you still remember how everything works.
We have a MySQL database that contains a node table, which lists every train station that is directly reachable from the Netherlands.
| id | name |
|---|---|
| 21392 | Nijmegen |
| 21598 | Nijmegen Dukenburg |
| 21601 | Nijmegen Heyendaal |
| 21603 | Nijmegen Lent |
| 21803 | Nijmegen Goffert |
Trains don’t just stop at train stations – they stop at specific platforms at those train stations. Those platforms are defined in the stop table.
| id | name | latitude | longitude | platform | node_id |
|---|---|---|---|---|---|
| 35025 | Nijmegen | 51.843352716672 | 5.8527283567859 | 1 | 21392 |
| 35026 | Nijmegen | 51.8440139658 | 5.85304677486 | 1a | 21392 |
| 35027 | Nijmegen | 51.842415589576 | 5.8522424471402 | 1b | 21392 |
| 35028 | Nijmegen | 51.843418123199 | 5.8525284242761 | 3 | 21392 |
| 35030 | Nijmegen | 51.8440338507 | 5.85280001163 | 3a | 21392 |
| 35031 | Nijmegen | 51.842471977321 | 5.8520218494775 | 3b | 21392 |
| 35032 | Nijmegen | 51.843382598179 | 5.8523051845771 | 4 | 21392 |
| 35033 | Nijmegen | 51.8440537357 | 5.85269272327 | 4a | 21392 |
| 35034 | Nijmegen | 51.842489994433 | 5.851883317404 | 4b | 21392 |
| 35029 | Nijmegen | 51.842214820937 | 5.8522482783764 | 35 | 21392 |
| 35035 | Nijmegen | 51.843845719325 | 5.8534115553016 | 21392 | |
| 35036 | Nijmegen | 51.842718896418 | 5.852064148637 | 21392 | |
| 47601 | Nijmegen | 51.8432285026 | 5.85303068161 | 21392 | |
| 35045 | Nijmegen Heyendaal | 51.8268250995 | 5.86774528027 | 1 | 21601 |
| 35046 | Nijmegen Heyendaal | 51.826879804 | 5.86788743734 | 2 | 21601 |
| 35047 | Nijmegen Heyendaal | 51.8266261734 | 5.86765944958 | 21601 | |
| 35048 | Nijmegen Heyendaal | 51.8266261734 | 5.86765944958 | 21601 |
Trains are scheduled to arrive at and depart from specific platforms at specific times. Once a train departs, it continues its trip towards the next stop. All this information can be found in the stop_time table.
| id | trip_id | stop_id | stop_sequence | arrival_time | departure_time | shape_dist_traveled |
|---|---|---|---|---|---|---|
| 25541 | 1807 | 35028 | 1 | 2023-06-01 15:28:00 | 2023-06-01 15:28:00 | 1 |
| 4395 | 1807 | 33353 | 5 | 2023-06-01 15:40:00 | 2023-06-01 15:45:00 | 18394 |
| 4396 | 1807 | 34041 | 8 | 2023-06-01 15:55:00 | 2023-06-01 15:55:00 | 34741 |
| 39696 | 1807 | 34705 | 9 | 2023-06-01 16:01:00 | 2023-06-01 16:01:00 | 41835 |
| 54327 | 1807 | 35518 | 14 | 2023-06-01 16:19:00 | 2023-06-01 16:26:00 | 75049 |
| 54328 | 1807 | 33511 | 20 | 2023-06-01 16:42:00 | 2023-06-01 16:44:00 | 104460 |
| 39697 | 1807 | 33569 | 22 | 2023-06-01 16:49:00 | 2023-06-01 16:51:00 | 110602 |
| 4397 | 1807 | 35346 | 23 | 2023-06-01 16:57:00 | 2023-06-01 16:57:00 | 119499 |
Passengers may need to change trains to reach their destination. Changing trains takes a non-zero amount of time. The transfer table tells us how much time is needed to transfer from one platform to another.
| id | from_stop_id | to_stop_id | min_transfer_time |
|---|---|---|---|
| 8612 | 35025 | 35036 | 240 |
| 8613 | 35025 | 35026 | 240 |
| 8614 | 35025 | 35027 | 240 |
| 4959 | 35026 | 35027 | 180 |
The final building block that we need for our journey planner is the result we get from querying from possible_path, which lists the best ways to travel from one station to another through the Dutch train network.
| path | transfers | stops | distance |
|---|---|---|---|
| 21601 – 21392 – 21680 – 21473 | 2 | 9 | 121928 |
| 21601 – 21392 – 21671 – 21473 | 2 | 11 | 131837 |
| 21601 – 21392 – 21678 – 21618 – 21473 | 3 | 8 | 114842 |
| 21601 – 21392 – 21251 – 21473 | 2 | 13 | 143748 |
| 21601 – 21392 – 21680 – 21618 – 21473 | 3 | 9 | 121927 |
Generating travel plans
As you may recall from the previous blog, we wrote a query that helps us navigate the train network from Nijmegen Heyendaal station to Amsterdam Science Park station. Now it’s time for us to pick up where we left off and generate some travel plans!
Before we start, let’s take a look at the website of the Dutch Railways. When we use its travel planner to generate plans for this route, we get the following result:
If we assume that our time of departure is 08:00, our best option is to take the 08:03 train from Nijmegen Heyendaal. After a few transfers on Nijmegen and Amsterdam Centraal, we should arrive at Amsterdam Science Park at 09:49.
Let’s try to replicate this advice using SQL by following the corresponding possible_path (21601 – 21392 – 21680 – 21473).
Dry run
I’ll first show how we can do this manually, step by step. The first step we need to take is to determine which trains depart from Nijmegen Heyendaal around 08:00. For simplicity’s sake, let’s assume that people are willing to spend at most 30 minutes waiting for a train.
We can use the following trip retrieval query to retrieve all departures from Nijmegen Heyendaal between 08:00 and 08:30:
SELECT trip_id, stop_sequence, departure_time, name, platform, stop_idFROM stop_timeINNER JOIN stop ON stop_id = stop.idWHERE node_id = 21601 -- Nijmegen Heyendaal AND departure_time BETWEEN '2023-06-01 08:00:00' AND '2023-06-01 09:00:00'ORDER BY departure_timeExecuting the query gives us eight results. Trains depart from this station every 15 minutes in both directions.
| trip_id | stop_sequence | departure_time | name | platform | stop_id |
|---|---|---|---|---|---|
| 5010 | 8 | 2023-06-01 08:03:00 | Nijmegen Heyendaal | 2 | 35046 |
| 2455 | 2 | 2023-06-01 08:11:00 | Nijmegen Heyendaal | 1 | 35045 |
| 4639 | 12 | 2023-06-01 08:18:00 | Nijmegen Heyendaal | 2 | 35046 |
| 684 | 2 | 2023-06-01 08:26:00 | Nijmegen Heyendaal | 1 | 35045 |
| 1389 | 6 | 2023-06-01 08:33:00 | Nijmegen Heyendaal | 2 | 35046 |
| 4641 | 2 | 2023-06-01 08:41:00 | Nijmegen Heyendaal | 1 | 35045 |
| 3218 | 12 | 2023-06-01 08:48:00 | Nijmegen Heyendaal | 2 | 35046 |
| 685 | 2 | 2023-06-01 08:56:00 | Nijmegen Heyendaal | 1 | 35045 |
The second station in our path was 21392 (Nijmegen), so let’s query the database to find out which trips from the previous search results can take us to Nijmegen:
SELECT stop_time.trip_id, stop_time.stop_sequence, stop_time.arrival_time, stop.name, stop.platform, stop_time.stop_idFROM stop_timeINNER JOIN stop ON stop_id = stop.idINNER JOIN ( -- These are the results from the previous query. We want to find out -- which of these trips take us to Nijmegen. SELECT 5010 AS trip_id, 8 AS stop_sequence UNION SELECT 2455 AS trip_id, 2 AS stop_sequence UNION SELECT 4639 AS trip_id, 12 AS stop_sequence UNION SELECT 684 AS trip_id, 2 AS stop_sequence UNION SELECT 1389 AS trip_id, 6 AS stop_sequence UNION SELECT 4641 AS trip_id, 2 AS stop_sequence UNION SELECT 3218 AS trip_id, 12 AS stop_sequence UNION SELECT 685 AS trip_id, 2 AS stop_sequence ) departing_stop ON -- If any of these trips passes by Nijmegen, this query should return -- `stop_time` records for Nijmegen `stop_id`s that include the same -- `trip_id`. departing_stop.trip_id = stop_time.trip_idWHERE node_id = 21392 -- Nijmegen AND -- Trains only go in one direction, so let’s make sure we only include trips -- that have not already stopped at Nijmegen. stop_time.stop_sequence > departing_stop.stop_sequenceORDER BY arrival_timeAs expected, only four out of those eight trips allow us to reach Nijmegen:
| trip_id | stop_sequence | arrival_time | name | platform | stop_id |
|---|---|---|---|---|---|
| 5010 | 9 | 2023-06-01 08:07:00 | Nijmegen | 35 | 35029 |
| 4639 | 13 | 2023-06-01 08:22:00 | Nijmegen | 1b | 35027 |
| 1389 | 7 | 2023-06-01 08:37:00 | Nijmegen | 35 | 35029 |
| 3218 | 13 | 2023-06-01 08:52:00 | Nijmegen | 1b | 35027 |
Once we are in Nijmegen, it’s time to change trains. We will want to retrieve all departures from this station again. Sadly, we can’t simply execute a modified version of the query that we just used for Nijmegen Heyendaal. Our journey no longer starts at “a station”, but at a specific platform (35 or 1b). Walking from one platform to another may take considerable time, especially at larger stations.
The trip retrieval query below retrieves all departures within the next 30 minutes that are realistically possible if one starts their journey from platforms 35 or 1b:
SELECT DISTINCT trip_id, stop_sequence, departure_time, name, platform, stop_idFROM stop_time INNER JOIN stop ON stop_id = stop.idLEFT JOIN transfer ON -- `stop_id`s for platforms 35 and 1b from the previous query from_stop_id IN (35027, 35029) AND to_stop_id = stop.idWHERE departure_time BETWEEN -- Earliest arrival time in Nijmegen (from previous query) + 30 minutes DATE_ADD('2023-06-01 08:07:00', INTERVAL min_transfer_time SECOND) AND DATE_ADD('2023-06-01 08:07:00', INTERVAL 30 MINUTE)ORDER BY departure_timeExecuting this query gives us the following results:
| trip_id | stop_sequence | departure_time | name | platform | stop_id | node_id |
|---|---|---|---|---|---|---|
| 3985 | 1 | 2023-06-01 08:13:00 | Nijmegen | 3 | 35028 | 21392 |
| 4017 | 19 | 2023-06-01 08:16:00 | Nijmegen | 1a | 35026 | 21392 |
| 3984 | 40 | 2023-06-01 08:17:00 | Nijmegen | 3 | 35028 | 21392 |
| 1122 | 5 | 2023-06-01 08:18:00 | Nijmegen | 4b | 35034 | 21392 |
| 4639 | 13 | 2023-06-01 08:22:00 | Nijmegen | 1b | 35027 | 21392 |
| 684 | 1 | 2023-06-01 08:23:00 | Nijmegen | 35 | 35029 | 21392 |
| 2634 | 17 | 2023-06-01 08:24:00 | Nijmegen | 1a | 35026 | 21392 |
| 3258 | 1 | 2023-06-01 08:28:00 | Nijmegen | 3 | 35028 | 21392 |
| 2628 | 12 | 2023-06-01 08:30:00 | Nijmegen | 4b | 35034 | 21392 |
| 1837 | 4 | 2023-06-01 08:32:00 | Nijmegen | 1a | 35026 | 21392 |
| 1802 | 23 | 2023-06-01 08:32:00 | Nijmegen | 3 | 35028 | 21392 |
| 1389 | 7 | 2023-06-01 08:37:00 | Nijmegen | 35 | 35029 | 21392 |
Planning the remainder of the trip should be a walk in the park. We can repeatedly modify and execute the last two trip retrieval queries with the remaining node_ids until we reach our final destination.
Stored procedures
You should now have a rough idea of how we can generate a travel plan by iteratively executing queries to retrieve departures and arrivals at each of the nodes in our path. If we were building an application using a normal programming language that’s probably exactly what we would implement in the final version, but that’s not what I’m going to do.
Instead, I’m going to write a stored procedure. A stored procedure is kind of like a function in a normal programming language. And since my manager seems to be very excited about this “big query thing, I am going to write one big-ass stored procedure that does everything. To hell with functional decomposition.
We’ll start with the following template:
DROP PROCEDURE IF EXISTS GET_PATH_ADVICE;
-- SQL statements are normally delimited using a semicolon. We’ll temporarily-- change the delimiter so we can use the semicolon character for statements-- within our stored procedure.DELIMITER $$
-- This creates a procedure GET_PATH_ADVICE() that accepts two parameters: a-- path (as computed using the possible_path CTE) and a time of departure.CREATE PROCEDURE GET_PATH_ADVICE( IN path VARCHAR(255), IN journey_departure_time DATETIME)BEGIN -- TODO: Declare variables so we can keep track of state
WHILE LENGTH(path) > 0 DO -- TODO: Set values for current iteration
IF step = 0 THEN -- TODO: Retrieve departures from first station ELSEIF step % 2 = 1 THEN -- TODO: Retrieve arrivals at intermediate and final stations ELSE -- TODO: Retrieve departures from intermediate stations END IF;
SET step = step + 1; END WHILE;
-- TODO: Generate output hereEND $$
DELIMITER ;Here we define a procedure GET_PATH_ADVICE, which given a path (as computed using the possible_path CTE) and a time of departure, will generate several travel plans in a WHILE loop, using modified versions of the three trip retrieval queries we saw earlier.
It doesn’t look very chonky yet, but that’s because it still contains a lot of TODOs that we need to process. In the remainder of this section, we’ll slowly feed it lines of SQL code until it’s morbidly obese.
Declaring variables
We begin by declaring some variables at the top of our procedure:
-
We use
nodeandstepto keep track of the current state.nodekeeps track of the station we are currently at, whilestepis a counter that we can use to determine which of the three trip retrieval queries we need to execute, among other things. -
A
destvariable makes it a bit easier for the algorithm to determine when it has created a travel plan for the complete journey (in this case from Nijmegen Heyendaal to Amsterdam Science Park). -
We also declare an
earliest_arrival_timedatetime. I’ll explain this one later, once we actually get to use it.
We also need a place to store intermediate and final query results. Although lists and maps (dictionaries) are not available in SQL, we can create a temporary table to store our data.
-- Scalar values can be declared using the DECLARE keyword. We’ll use these-- to keep track of where we are in the `path`.DECLARE node VARCHAR(255);DECLARE step SMALLINT DEFAULT 0;-- We’re saving these for later.DECLARE dest VARCHAR(255);DECLARE earliest_arrival_time DATETIME;
-- Lists aren’t a thing in SQL, so if we want to store lists of things we’ll-- need a temporary table.CREATE TEMPORARY TABLE IF NOT EXISTS tmp_path_advice ( stop_times TEXT, cur_step SMALLINT, cur_stop_id INTEGER, cur_trip_id INTEGER, cur_stop_sequence SMALLINT, first_datetime DATETIME, cur_datetime DATETIME, is_complete BOOLEAN DEFAULT FALSE);dest is used to determine whether a travel plan is “complete”. We set its value to the last node_id in the path string that is passed to the GET_PATH_ADVICE procedure.
-- Extract the last `node_id` from the `path` string.SET dest = SUBSTRING_INDEX(SUBSTRING_INDEX(path, ' – ', -1), ' – ', 1);The node variable is set in the first and every other iteration of the WHILE loop, i.e. when a traveller starts their journey at the first station or reaches a new station by train. When this happens, we also remove the current node from the path.
-- The `node` and `path` variables keep track of the current state of-- the algorithm. These only need to be set when a traveller reaches a-- new station, i.e. when they depart from their origin and when they-- arrive at another station.IF step = 0 OR step % 2 = 1 THEN SET node = SUBSTRING_INDEX(path, ' – ', 1); SET path = SUBSTRING(path, LENGTH(node) + 3); SET node = TRIM(node);END IF;Departure from first station
The query that retrieves all departures from the origin station is based on the first of our three trip retrieval queries. I’ve made a few modifications to it. For starters, nodeand departure_time are no longer hardcoded, but determined dynamically. We also fetch different fields, which we insert into the tmp_path_advice table:
-
stop_timesstores references to relevantstop_timerecords, which contain all the information we need to generate human-friendly travel advice. -
cur_stepstores the number of the current iteration. This allows the algorithm to properly appendstop_time.ids to thestop_timesfield. -
cur_stop_idis used to determine whichtrips can be taken from a transfer station (like Nijmegen). -
cur_trip_idandcur_stop_sequencecan be used to determine whichtrips can be taken to arrive at the nextnode. -
first_datetimeis when someone boards a train for the first time. This value will stay the same in subsequent iterations. -
cur_datetimeis either the current departure time or arrival time. This is used to determine which transfers are possible, among other things. -
is_completeis a boolean value that tells us whether the computedstop_timesis complete. It’s always false here, because we haven’t even left the first station at this point.
INSERT INTO tmp_path_adviceSELECT -- These are all initial values stop_time.id, -- stop_times step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence departure_time, -- first_datetime departure_time, -- cur_datetime FALSE -- is_completeFROM stop_timeINNER JOIN stop ON stop_id = stop.idWHERE node_id = node AND departure_time BETWEEN -- Retrieve all trains that depart in the next 30 minutes journey_departure_time AND DATE_ADD(journey_departure_time, INTERVAL 30 MINUTE)ORDER BY departure_time;After the first iteration, the tmp_path_advice table looks roughly like this:
| stop_times | cur_step | cur_stop_id | cur_trip_id | cur_stop_sequence | first_datetime | cur_datetime | is_complete |
|---|---|---|---|---|---|---|---|
| 7352 | 0 | 35046 | 5010 | 8 | 2023-06-01 08:03:00 | 2023-06-01 08:03:00 | 0 |
| 29055 | 0 | 35045 | 2455 | 2 | 2023-06-01 08:11:00 | 2023-06-01 08:11:00 | 0 |
| 36810 | 0 | 35046 | 4639 | 12 | 2023-06-01 08:18:00 | 2023-06-01 08:18:00 | 0 |
| 21877 | 0 | 35045 | 684 | 2 | 2023-06-01 08:26:00 | 2023-06-01 08:26:00 | 0 |
Arrivals at subsequent stations
In the next iteration of the while loop, the procedure will attempt to find all stations that can be reached with the trips from the iteration directly before it. It then inserts a record for each of the reachable stops:
INSERT INTO tmp_path_adviceSELECT CONCAT(stop_times, ',', stop_time.id), -- stop_times step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence first_datetime, -- first_datetime stop_time.arrival_time, -- cur_datetime node_id = dest -- is_completeFROM tmp_path_adviceINNER JOIN stop_time ON -- The train has already stopped at stations with a lower -- stop_sequence, so we’ll need to ignore those. stop_time.trip_id = cur_trip_id AND stop_time.stop_sequence > cur_stop_sequenceINNER JOIN stop ON stop_id = stop.idWHERE cur_step = step - 1 AND stop_times IS NOT NULLORDER BY arrival_time;The tmp_path_advice table now has 18 new rows, with each row representing a stop that can be reached via one of the four trips at a specific time:
| stop_times | cur_step | cur_stop_id | cur_trip_id | cur_stop_sequence | first_datetime | cur_datetime | is_complete |
|---|---|---|---|---|---|---|---|
| 7352 | 0 | 35046 | 5010 | 8 | 2023-06-01 08:03:00 | 2023-06-01 08:03:00 | 0 |
| 29055 | 0 | 35045 | 2455 | 2 | 2023-06-01 08:11:00 | 2023-06-01 08:11:00 | 0 |
| 36810 | 0 | 35046 | 4639 | 12 | 2023-06-01 08:18:00 | 2023-06-01 08:18:00 | 0 |
| 21877 | 0 | 35045 | 684 | 2 | 2023-06-01 08:26:00 | 2023-06-01 08:26:00 | 0 |
| 7352,42562 | 1 | 35029 | 5010 | 9 | 2023-06-01 08:03:00 | 2023-06-01 08:07:00 | 0 |
| 29055,15802 | 1 | 34949 | 2455 | 3 | 2023-06-01 08:11:00 | 2023-06-01 08:16:00 | 0 |
| 29055,36136 | 1 | 33838 | 2455 | 4 | 2023-06-01 08:11:00 | 2023-06-01 08:22:00 | 0 |
| 36810,16415 | 1 | 35027 | 4639 | 13 | 2023-06-01 08:18:00 | 2023-06-01 08:22:00 | 0 |
| 29055,8394 | 1 | 33732 | 2455 | 5 | 2023-06-01 08:11:00 | 2023-06-01 08:29:00 | 0 |
| 21877,35446 | 1 | 34949 | 684 | 3 | 2023-06-01 08:26:00 | 2023-06-01 08:32:00 | 0 |
| 21877,15203 | 1 | 33838 | 684 | 4 | 2023-06-01 08:26:00 | 2023-06-01 08:36:00 | 0 |
| 29055,36137 | 1 | 35626 | 2455 | 6 | 2023-06-01 08:11:00 | 2023-06-01 08:37:00 | 0 |
| 29055,50913 | 1 | 35643 | 2455 | 7 | 2023-06-01 08:11:00 | 2023-06-01 08:43:00 | 0 |
| 21877,42934 | 1 | 33732 | 684 | 5 | 2023-06-01 08:26:00 | 2023-06-01 08:44:00 | 0 |
| 21877,15204 | 1 | 35626 | 684 | 6 | 2023-06-01 08:26:00 | 2023-06-01 08:51:00 | 0 |
| 29055,8395 | 1 | 33768 | 2455 | 8 | 2023-06-01 08:11:00 | 2023-06-01 08:57:00 | 0 |
| 21877,105 | 1 | 35645 | 684 | 7 | 2023-06-01 08:26:00 | 2023-06-01 08:57:00 | 0 |
| 29055,29056 | 1 | 35621 | 2455 | 9 | 2023-06-01 08:11:00 | 2023-06-01 09:01:00 | 0 |
| 29055,29057 | 1 | 35456 | 2455 | 10 | 2023-06-01 08:11:00 | 2023-06-01 09:07:00 | 0 |
| 29055,36138 | 1 | 35293 | 2455 | 11 | 2023-06-01 08:11:00 | 2023-06-01 09:14:00 | 0 |
| 29055,773 | 1 | 35359 | 2455 | 12 | 2023-06-01 08:11:00 | 2023-06-01 09:20:00 | 0 |
| 29055,50914 | 1 | 35213 | 2455 | 13 | 2023-06-01 08:11:00 | 2023-06-01 09:25:00 | 0 |
Departures from subsequent stations
The third trip retrieval query, which retrieves departures for subsequent stations on a journey, is similarly structured:
INSERT INTO tmp_path_adviceSELECT DISTINCT CONCAT(stop_times, ',', stop_time.id), -- stop_times step AS step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence first_datetime, -- first_datetime departure_time, -- cur_datetime node_id = dest -- is_completeFROM tmp_path_adviceINNER JOIN stop ON node_id = nodeINNER JOIN stop_time ON stop.id = stop_idLEFT JOIN transfer ON from_stop_id = cur_stop_id AND to_stop_id = stop_idWHERE cur_step = step - 1 AND stop_times IS NOT NULL AND -- This is the Netherlands, so we’re not willing to consider -- “transfers” that take more than half an hour. departure_time BETWEEN DATE_ADD(cur_datetime, INTERVAL min_transfer_time SECOND) AND DATE_ADD(cur_datetime, INTERVAL 30 MINUTE)ORDER BY departure_time;This query inserts the following records:
| stop_times | cur_step | cur_stop_id | cur_trip_id | cur_stop_sequence | first_datetime | cur_datetime | is_complete |
|---|---|---|---|---|---|---|---|
| 7352,42562,53344 | 2 | 35028 | 3985 | 1 | 2023-06-01 08:03:00 | 2023-06-01 08:13:00 | 0 |
| 7352,42562,53354 | 2 | 35026 | 4017 | 19 | 2023-06-01 08:03:00 | 2023-06-01 08:16:00 | 0 |
| 7352,42562,29492 | 2 | 35028 | 3984 | 40 | 2023-06-01 08:03:00 | 2023-06-01 08:17:00 | 0 |
| 7352,42562,32231 | 2 | 35034 | 1122 | 5 | 2023-06-01 08:03:00 | 2023-06-01 08:18:00 | 0 |
| 7352,42562,16415 | 2 | 35027 | 4639 | 13 | 2023-06-01 08:03:00 | 2023-06-01 08:22:00 | 0 |
| 7352,42562,35445 | 2 | 35029 | 684 | 1 | 2023-06-01 08:03:00 | 2023-06-01 08:23:00 | 0 |
| 7352,42562,17518 | 2 | 35026 | 2634 | 17 | 2023-06-01 08:03:00 | 2023-06-01 08:24:00 | 0 |
| 7352,42562,26028 | 2 | 35028 | 3258 | 1 | 2023-06-01 08:03:00 | 2023-06-01 08:28:00 | 0 |
| 36810,16415,26028 | 2 | 35028 | 3258 | 1 | 2023-06-01 08:18:00 | 2023-06-01 08:28:00 | 0 |
| 7352,42562,51057 | 2 | 35034 | 2628 | 12 | 2023-06-01 08:03:00 | 2023-06-01 08:30:00 | 0 |
| 36810,16415,51057 | 2 | 35034 | 2628 | 12 | 2023-06-01 08:18:00 | 2023-06-01 08:30:00 | 0 |
| 7352,42562,30742 | 2 | 35026 | 1837 | 4 | 2023-06-01 08:03:00 | 2023-06-01 08:32:00 | 0 |
| 7352,42562,10068 | 2 | 35028 | 1802 | 23 | 2023-06-01 08:03:00 | 2023-06-01 08:32:00 | 0 |
| 36810,16415,10068 | 2 | 35028 | 1802 | 23 | 2023-06-01 08:18:00 | 2023-06-01 08:32:00 | 0 |
| 36810,16415,30742 | 2 | 35026 | 1837 | 4 | 2023-06-01 08:18:00 | 2023-06-01 08:32:00 | 0 |
| 7352,42562,15424 | 2 | 35029 | 1389 | 7 | 2023-06-01 08:03:00 | 2023-06-01 08:37:00 | 0 |
| 36810,16415,15424 | 2 | 35029 | 1389 | 7 | 2023-06-01 08:18:00 | 2023-06-01 08:37:00 | 0 |
| 36810,16415,19736 | 2 | 35027 | 4641 | 1 | 2023-06-01 08:18:00 | 2023-06-01 08:38:00 | 0 |
| 36810,16415,41152 | 2 | 35033 | 379 | 16 | 2023-06-01 08:18:00 | 2023-06-01 08:39:00 | 0 |
| 36810,16415,26193 | 2 | 35028 | 3865 | 1 | 2023-06-01 08:18:00 | 2023-06-01 08:43:00 | 0 |
| 36810,16415,4978 | 2 | 35026 | 3289 | 19 | 2023-06-01 08:18:00 | 2023-06-01 08:46:00 | 0 |
| 36810,16415,32927 | 2 | 35028 | 3247 | 40 | 2023-06-01 08:18:00 | 2023-06-01 08:47:00 | 0 |
| 36810,16415,56419 | 2 | 35034 | 2544 | 5 | 2023-06-01 08:18:00 | 2023-06-01 08:48:00 | 0 |
| 36810,16415,21007 | 2 | 35027 | 3218 | 13 | 2023-06-01 08:18:00 | 2023-06-01 08:52:00 | 0 |
These two INSERT queries are repeated until the procedure has reached the end of the path. When this happens, the tmp_path_advice table hopefully has some records where is_complete is truthy!
Data cleanup
Once the while loop terminates, all that remains is to generate output that can be displayed to a user. But first we need to clean up some data.
Let’s start by removing all records from tmp_path_advice that we don’t need:
-- Get rid of all partial trip advice. We only need the final results.DELETE tmp_path_adviceFROM tmp_path_adviceWHERE is_complete = FALSE;We then retrieve the earliest possible time of arrival and store it in the earliest_arrival_time variable. The best travel plans are those that have a cur_datetime equal to earliest_arrival_time.
-- Determine the earliest possible time of arrival across all possible-- journeys.SET earliest_arrival_time = ( SELECT MIN(cur_datetime) FROM tmp_path_advice);A user may also be interested in alternative travel plans, e.g. plans that suggest taking a later train (keeping the overall travel time the same) or adding a bit more slack during transfers (increasing the overall travel time). However, what we would like to avoid are plans where a traveller would do this at every station along the path. Therefore, we only consider travel plans where the arrival time at the final destination is at most thirty minutes later than the earliest_arrival_time.
-- Discard all travel advice that takes more than 30 minutes longer than the-- fastest possible journey.DELETE tmp_path_adviceFROM tmp_path_adviceWHERE cur_datetime > DATE_ADD(earliest_arrival_time, INTERVAL 30 MINUTE);Generating advice
Now we can generate our travel advice. The tmp_path_advice table now only contains records that look a lot like this one here:
| stop_times | cur_step | cur_stop_id | cur_trip_id | cur_stop_sequence | first_datetime | cur_datetime | is_complete |
|---|---|---|---|---|---|---|---|
| 7352,42562,53344,31435,43651,850 | 5 | 33602 | 2209 | 7 | 2023-06-01 08:03:00 | 2023-06-01 09:49:00 | 1 |
The stop_times fields contains stop_time.ids that can be used to look up almost all the information that we want to show to users. Unfortunately this field is a comma-separated list, so we can’t JOIN on this field directly. However, what we can do is convert stop_times values to a JSON list, parse it, and then CROSS JOIN with the resulting table. This allows us to simply SELECT the information we need from the joined result:
-- This SELECT query generates the human-readable output of this procedure.SELECT DISTINCT SUBSTRING(SHA1(stop_times), 1, 8) AS journey, IF(sequence % 2 = 1, 'Departure', 'Arrival') AS type, TIME(IF(sequence % 2 = 1, departure_time, arrival_time)) AS time, stop.name AS station, stop.platform, CONCAT(agency.name, ' ', trip.long_name) AS train_type, trip.headsign, first_datetime AS journey_departure, cur_datetime AS journey_arrivalFROM tmp_path_adviceCROSS JOIN -- `stop_times` is a comma-separated string. By converting it to a JSON -- list we can iterate over it and extract the key (`sequence`) and -- `value` of each element. JSON_TABLE ( CONCAT('[', stop_times, ']'), '$[*]' COLUMNS ( sequence FOR ORDINALITY, value INT PATH '$' ) ) resultINNER JOIN stop_time ON result.value = stop_time.idINNER JOIN stop ON stop_id = stop.idINNER JOIN trip ON trip_id = trip.idINNER JOIN route ON route_id = route.idINNER JOIN agency ON agency_id = agency.idWHERE is_completeORDER BY -- We want to arrive at our destination as early as possible. When -- multiple options have the same arrival time, prefer options that take -- less time, i.e. have a later time of departure. cur_datetime, first_datetime DESC, journey, -- Used to keep the rows for each journey together sequence; -- Keep rows within journeys in the right orderPutting it all together
Now that everything is in place, it’s time to show the complete version of the GET_PATH_ADVICE procedure. The text continues below this snippet. Click here if you don’t feel like scrolling until your finger falls off.
DROP PROCEDURE IF EXISTS GET_PATH_ADVICE;
-- SQL statements are normally delimited using a semicolon. We’ll temporarily-- change the delimiter so we can use the semicolon character for statements-- within our stored procedure.DELIMITER $$
-- This creates a procedure GET_PATH_ADVICE() that accepts two parameters: a-- path (as computed using the possible_path CTE) and the time of departure.CREATE PROCEDURE GET_PATH_ADVICE( IN path VARCHAR(255), IN journey_departure_time DATETIME)BEGIN -- We’ll start by declaring some variables.
-- Scalar values can be declared using the DECLARE keyword. We’ll use these -- to keep track of where we are in the `path`. DECLARE node VARCHAR(255); DECLARE step SMALLINT DEFAULT 0; -- We’re saving these for later. DECLARE dest VARCHAR(255); DECLARE earliest_arrival_time DATETIME;
-- Lists aren’t a thing in SQL, so if we want to store lists of things we’ll -- need a temporary table. CREATE TEMPORARY TABLE IF NOT EXISTS tmp_path_advice ( stop_times TEXT, cur_step SMALLINT, cur_stop_id INTEGER, cur_trip_id INTEGER, cur_stop_sequence SMALLINT, first_datetime DATETIME, cur_datetime DATETIME, is_complete BOOLEAN DEFAULT FALSE );
-- Extract the last `node_id` from the `path` string. SET dest = SUBSTRING_INDEX(SUBSTRING_INDEX(path, ' – ', -1), ' – ', 1);
-- Iterate through the given path until we’ve reached the end. WHILE LENGTH(path) > 0 DO -- The `node` and `path` variables keep track of the current state of -- the algorithm. These only need to be set when a traveller reaches a -- new station, i.e. when they depart from their origin and when they -- arrive at another station. IF step = 0 OR step % 2 = 1 THEN SET node = SUBSTRING_INDEX(path, ' – ', 1); SET path = SUBSTRING(path, LENGTH(node) + 3); SET node = TRIM(node); END IF;
-- The first leg of the journey is special, because we do not start at a -- specific platform and thus also do not have to take transfer times -- into account. IF step = 0 THEN INSERT INTO tmp_path_advice SELECT -- These are all initial values stop_time.id, -- stop_times step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence departure_time, -- first_datetime departure_time, -- cur_datetime FALSE -- is_complete FROM stop_time INNER JOIN stop ON stop_id = stop.id WHERE node_id = node AND departure_time BETWEEN -- Retrieve all trains that depart in the next 30 minutes journey_departure_time AND DATE_ADD(journey_departure_time, INTERVAL 30 MINUTE) ORDER BY departure_time; -- Arrivals at new stations are handled as part of odd steps. ELSEIF step % 2 = 1 THEN INSERT INTO tmp_path_advice SELECT CONCAT(stop_times, ',', stop_time.id), -- stop_times step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence first_datetime, -- first_datetime stop_time.arrival_time, -- cur_datetime node_id = dest -- is_complete FROM tmp_path_advice INNER JOIN stop_time ON -- The train has already stopped at stations with a lower -- stop_sequence, so we’ll need to ignore those. stop_time.trip_id = cur_trip_id AND stop_time.stop_sequence > cur_stop_sequence INNER JOIN stop ON stop_id = stop.id WHERE cur_step = step - 1 AND stop_times IS NOT NULL ORDER BY arrival_time; ELSE -- Departures from stations are handled as part of even steps. INSERT INTO tmp_path_advice SELECT DISTINCT CONCAT(stop_times, ',', stop_time.id), -- stop_times step AS step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence first_datetime, -- first_datetime departure_time, -- cur_datetime node_id = dest -- is_complete FROM tmp_path_advice INNER JOIN stop ON node_id = node INNER JOIN stop_time ON stop.id = stop_id LEFT JOIN transfer ON from_stop_id = cur_stop_id AND to_stop_id = stop_id WHERE cur_step = step - 1 AND stop_times IS NOT NULL AND -- This is the Netherlands, so we’re not willing to consider -- “transfers” that take more than half an hour. departure_time BETWEEN DATE_ADD(cur_datetime, INTERVAL min_transfer_time SECOND) AND DATE_ADD(cur_datetime, INTERVAL 30 MINUTE) ORDER BY departure_time; END IF;
SET step = step + 1; END WHILE;
-- Get rid of all partial trip advice. We only need the final results. DELETE tmp_path_advice FROM tmp_path_advice WHERE is_complete = FALSE;
-- Determine the earliest possible time of arrival across all possible -- journeys. SET earliest_arrival_time = ( SELECT MIN(cur_datetime) FROM tmp_path_advice );
-- Discard all travel advice that takes more than 30 minutes longer than the -- fastest possible journey. DELETE tmp_path_advice FROM tmp_path_advice WHERE cur_datetime > DATE_ADD(earliest_arrival_time, INTERVAL 30 MINUTE);
-- This SELECT query generates the human-readable output of this procedure. SELECT DISTINCT SUBSTRING(SHA1(stop_times), 1, 8) AS journey, IF(sequence % 2 = 1, 'Departure', 'Arrival') AS type, TIME(IF(sequence % 2 = 1, departure_time, arrival_time)) AS time, stop.name AS station, stop.platform, CONCAT(agency.name, ' ', trip.long_name) AS train_type, trip.headsign, first_datetime AS journey_departure, cur_datetime AS journey_arrival FROM tmp_path_advice CROSS JOIN -- `stop_times` is a comma-separated string. By converting it to a JSON -- list we can iterate over it and extract the key (`sequence`) and -- `value` of each element. JSON_TABLE ( CONCAT('[', stop_times, ']'), '$[*]' COLUMNS ( sequence FOR ORDINALITY, value INT PATH '$' ) ) result INNER JOIN stop_time ON result.value = stop_time.id INNER JOIN stop ON stop_id = stop.id INNER JOIN trip ON trip_id = trip.id INNER JOIN route ON route_id = route.id INNER JOIN agency ON agency_id = agency.id WHERE is_complete ORDER BY -- We want to arrive at our destination as early as possible. When -- multiple options have the same arrival time, prefer options that take -- less time, i.e. have a later time of departure. cur_datetime, first_datetime DESC, journey, -- Used to keep the rows for each journey together sequence; -- Keep rows within journeys in the right order
-- Now that we’ve generated the output we don’t need this any more! DROP TEMPORARY TABLE IF EXISTS tmp_path_advice;END $$
DELIMITER ;We can call this stored procedure as follows:
CALL GET_PATH_ADVICE('21601 – 21392 – 21680 – 21473', '2023-06-01 08:00:00');The table below displays the results of the procedure call. I have excluded the last two columns to ensure that the table fits on the page. I have also given each travel plan a unique colour to make it easier to see where each journey starts and ends.
| journey | type | time | station | platform | train_type | headsign |
|---|---|---|---|---|---|---|
| 8eee2c55 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen |
| 8eee2c55 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen |
| 8eee2c55 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder |
| 8eee2c55 | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder |
| 8eee2c55 | Departure | 09:41:00 | Amsterdam Centraal | 13a | NS Sprinter | Amersfoort Vathorst |
| 8eee2c55 | Arrival | 09:49:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst |
| ab329c3d | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen |
| ab329c3d | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen |
| ab329c3d | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder |
| ab329c3d | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder |
| ab329c3d | Departure | 09:53:00 | Amsterdam Centraal | 10a | NS Sprinter | Zwolle |
| ab329c3d | Arrival | 10:01:00 | Amsterdam Science Park | 1 | NS Sprinter | Zwolle |
| 0ece0c64 | Departure | 08:18:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen |
| 0ece0c64 | Arrival | 08:22:00 | Nijmegen | 1b | Arriva Stoptrein | Nijmegen |
| 0ece0c64 | Departure | 08:43:00 | Nijmegen | 3 | NS Intercity | Den Helder |
| 0ece0c64 | Arrival | 10:05:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder |
| 0ece0c64 | Departure | 10:11:00 | Amsterdam Centraal | 13a | NS Sprinter | Amersfoort Vathorst |
| 0ece0c64 | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst |
Really putting it all together
We can’t pat ourselves on the back yet: our GET_PATH_ADVICE() only works for one path, while the possible_path common table expression gave us five different paths to explore. What we want is a single procedure that is almost as easy to use as the official Dutch Railways travel planner. We’ll have to combine the possible_path query with GET_PATH_ADVICE() into one gargantuan procedure that does everything.
The resulting procedure is listed below. I’ve added a few explanatory comments here and there. Needless to say, it’s long, so feel free to click here to skip to the end.
DROP PROCEDURE IF EXISTS PLAN_JOURNEY;
DELIMITER $$
CREATE PROCEDURE PLAN_JOURNEY( IN origin VARCHAR(255), -- Station name IN destination VARCHAR(255), -- Station name IN journey_departure_time DATETIME -- Just a datetime)BEGIN -- ---------------------------------------------------------------------- -- -- VARIABLE DECLARATIONS -- -- ---------------------------------------------------------------------- -- DECLARE orig_node INTEGER; DECLARE dest_node INTEGER; DECLARE done BOOLEAN; DECLARE path VARCHAR(255);
-- These variables are used for the algorithm that looks up departures and -- arrivals in `stop_time` for each `path` DECLARE node VARCHAR(255); DECLARE step SMALLINT DEFAULT 0; DECLARE dest VARCHAR(255); DECLARE earliest_arrival_time DATETIME;
-- By declaring a cursor we can iterate over the results of this common -- table expression. DECLARE best_paths_cursor CURSOR FOR WITH RECURSIVE allowed_node (node_id) AS ( SELECT DISTINCT node_id FROM stop INNER JOIN ( SELECT -- 0.25 latitude and 0.20 longitude should be roughly -- equivalent to 22 kilometres in the Netherlands MIN(latitude) - 0.25 AS min_lat, MAX(latitude) + 0.25 AS max_lat, MIN(longitude) - 0.25 AS min_lon, MAX(longitude) + 0.25 AS max_lon FROM stop WHERE node_id IN ( orig_node, dest_node ) ) bounding_box ON latitude BETWEEN min_lat AND max_lat AND longitude BETWEEN min_lon AND max_lon AND -- This is a stupid hack to remove anything that’s not -- clearly a train station, as bus/tram/metro stops in this -- dataset tend to have names that include both a place and -- a street name, e.g. “Amsterdam, Rokin”. name NOT LIKE '%,%' ), possible_path (nodes, transfers, stops, distance, current_node) AS ( -- Base case SELECT CONCAT(' – ', CAST(from_node_id AS CHAR(255)), ' – '), 0, stops, distance, to_node_id FROM edge WHERE from_node_id = orig_node AND to_node_id IN (SELECT node_id FROM allowed_node)
UNION ALL
-- Recursive case SELECT CONCAT(nodes, from_node_id, ' – '), transfers + 1, possible_path.stops + edge.stops, possible_path.distance + edge.distance, to_node_id FROM possible_path INNER JOIN edge ON current_node = from_node_id WHERE -- Only consider nodes within that rectangular bounding box -- that we discussed earlier. to_node_id IN (SELECT node_id FROM allowed_node) AND -- Used to detect cycles within a possible path. It makes no -- sense to visit the same station twice! nodes NOT LIKE CONCAT('% – ', to_node_id, ' – %') AND -- This basically sets the number of “layers” that our -- pathfinding algorithm is allowed to explore. With -- transfers set to 3, the execution time is roughly a -- second on my machine. With 4 it’s about 20 seconds. I -- haven’t tried to set it to 5, but it’ll probably take -- *very* long to complete. transfers < 3 ) SELECT DISTINCT -- The SUBSTRING() is used to remove the leading “ – ” that we added -- in the base case. SUBSTRING(CONCAT(nodes, current_node), 4) AS path FROM possible_path WHERE current_node = dest_node ORDER BY -- This seems to work reasonably well as a way to score different -- paths. The basic idea is that direct connections (fewer -- transfers) are preferred over journeys with many transfers, and -- intercity/express trains (fewer stops) are preferred over slower, -- local trains (many stops). transfers * stops, distance LIMIT 5; -- Completely arbitrary limit
-- Make sure that algorithm terminates DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
-- ---------------------------------------------------------------------- -- -- INPUT VALIDATION -- -- ---------------------------------------------------------------------- --
-- Retrieve `node.id`s for departure and arrival stations SET orig_node = ( SELECT id FROM node WHERE name COLLATE utf8mb4_unicode_ci = origin ); SET dest_node = ( SELECT id FROM node WHERE name COLLATE utf8mb4_unicode_ci = destination );
-- Abort procedure with a descriptive message if something is wrong with the -- input. IF orig_node IS NULL THEN SIGNAL SQLSTATE '02404' SET MESSAGE_TEXT = 'Origin node not found.'; END IF;
IF dest_node IS NULL THEN SIGNAL SQLSTATE '02404' SET MESSAGE_TEXT = 'Destination node not found.'; END IF;
IF origin = destination THEN SIGNAL SQLSTATE '02400' SET MESSAGE_TEXT = 'Origin and destination are the same.'; END IF;
-- ---------------------------------------------------------------------- -- -- GENERATE ADVICE -- -- ---------------------------------------------------------------------- --
-- This table will be used to store the final results CREATE TEMPORARY TABLE IF NOT EXISTS travel_advice ( journey VARCHAR(255), type VARCHAR(255), time DATETIME, station VARCHAR(255), platform VARCHAR(255), train_type VARCHAR(255), headsign VARCHAR(255), departure DATETIME, arrival DATETIME,
PRIMARY KEY (journey, time) );
OPEN best_paths_cursor;
read_loop: LOOP -- We look up timetables for each `possible_path`. Intermediate results -- are stored per path in `tmp_path_advice`. CREATE TEMPORARY TABLE IF NOT EXISTS tmp_path_advice ( stop_times TEXT, cur_step SMALLINT, cur_stop_id INTEGER, cur_trip_id INTEGER, cur_stop_sequence SMALLINT, first_datetime DATETIME, cur_datetime DATETIME, is_complete BOOLEAN DEFAULT FALSE );
FETCH best_paths_cursor INTO path;
IF done THEN LEAVE read_loop; END IF;
SET step = 0; SET earliest_arrival_time = NULL; -- Extract the last `node_id` from the `path` string. SET dest = SUBSTRING_INDEX(SUBSTRING_INDEX(path, ' – ', -1), ' – ', 1);
WHILE LENGTH(path) > 0 DO -- The `node` and `path` variables keep track of the current -- state of the algorithm. These only need to be set when a -- traveller reaches a new station, i.e. when they depart from -- their origin and when they arrive at another station. IF step = 0 OR step % 2 = 1 THEN SET node = SUBSTRING_INDEX(path, ' – ', 1); SET path = SUBSTRING(path, LENGTH(node) + 3); SET node = TRIM(node); END IF;
-- The first leg of the journey is special, because we do not -- start at a specific platform and thus also do not have to -- take transfer times into account. IF step = 0 THEN INSERT INTO tmp_path_advice SELECT -- These are all initial values stop_time.id, -- stop_times step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequence departure_time, -- first_datetime departure_time, -- cur_datetime FALSE -- is_complete FROM stop_time INNER JOIN stop ON stop_id = stop.id WHERE node_id = node AND departure_time BETWEEN -- Retrieve all trains that depart in the next 30 -- minutes journey_departure_time AND DATE_ADD(journey_departure_time, INTERVAL 30 MINUTE) ORDER BY departure_time; -- Arrivals at new stations are handled as part of odd steps. ELSEIF step % 2 = 1 THEN INSERT INTO tmp_path_advice SELECT CONCAT(stop_times, ',', stop_time.id), -- stop_times step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequ… first_datetime, -- first_datetim… stop_time.arrival_time, -- cur_datetime node_id = dest -- is_complete FROM tmp_path_advice INNER JOIN stop_time ON -- The train has already stopped at stations with a -- lower stop_sequence, so we’ll need to ignore -- those. stop_time.trip_id = cur_trip_id AND stop_time.stop_sequence > cur_stop_sequence INNER JOIN stop ON stop_id = stop.id WHERE cur_step = step - 1 AND stop_times IS NOT NULL ORDER BY arrival_time; ELSE -- Departures from stations are handled as part of even -- steps. INSERT INTO tmp_path_advice SELECT DISTINCT CONCAT(stop_times, ',', stop_time.id), -- stop_times step AS step, -- cur_step stop_id, -- cur_stop_id trip_id, -- cur_trip_id stop_sequence, -- cur_stop_sequ… first_datetime, -- first_datetim… departure_time, -- cur_datetime node_id = dest -- is_complete FROM tmp_path_advice INNER JOIN stop ON node_id = node INNER JOIN stop_time ON stop.id = stop_id LEFT JOIN transfer ON from_stop_id = cur_stop_id AND to_stop_id = stop_id WHERE cur_step = step - 1 AND stop_times IS NOT NULL AND -- This is the Netherlands, so we’re not willing to -- consider “transfers” that take more than half an -- hour. departure_time BETWEEN DATE_ADD( cur_datetime, INTERVAL min_transfer_time SECOND ) AND DATE_ADD(cur_datetime, INTERVAL 30 MINUTE) ORDER BY departure_time; END IF;
SET step = step + 1; END WHILE;
-- Get rid of all partial trip advice. We only need the final -- results. DELETE tmp_path_advice FROM tmp_path_advice WHERE is_complete = FALSE;
-- Determine the earliest possible time of arrival across all -- possible journeys. SET earliest_arrival_time = ( SELECT MIN(cur_datetime) FROM tmp_path_advice );
-- Discard all travel advice that takes more than 30 minutes longer -- than the fastest possible journey. DELETE tmp_path_advice FROM tmp_path_advice WHERE cur_datetime > DATE_ADD( earliest_arrival_time, INTERVAL 30 MINUTE );
-- This SELECT query generates the human-readable output of this -- procedure. INSERT IGNORE INTO travel_advice SELECT DISTINCT SUBSTRING(SHA1(stop_times), 1, 8) AS journey, IF(sequence % 2 = 1, 'Departure', 'Arrival') AS type, IF(sequence % 2 = 1, departure_time, arrival_time) AS time, stop.name AS station, stop.platform, CONCAT(agency.name, ' ', trip.long_name) AS train_type, trip.headsign, first_datetime AS journey_departure, cur_datetime AS journey_arrival FROM tmp_path_advice CROSS JOIN -- `stop_times` is a comma-separated string. By converting it -- to a JSON list we can iterate over it and extract the key -- (`sequence`) and `value` of each element. JSON_TABLE ( CONCAT('[', stop_times, ']'), '$[*]' COLUMNS ( sequence FOR ORDINALITY, value INT PATH '$' ) ) result INNER JOIN stop_time ON result.value = stop_time.id INNER JOIN stop ON stop_id = stop.id INNER JOIN trip ON trip_id = trip.id INNER JOIN route ON route_id = route.id INNER JOIN agency ON agency_id = agency.id WHERE is_complete ORDER BY -- We want to arrive at our destination as early as possible. -- When multiple options have the same arrival time, prefer -- options that take less time, i.e. have a later time of -- departure. cur_datetime, first_datetime DESC, journey, -- Used to keep the rows for each journey together sequence; -- Keep rows within journeys in the right order
DROP TEMPORARY TABLE IF EXISTS tmp_path_advice;
END LOOP;
CLOSE best_paths_cursor;
-- ---------------------------------------------------------------------- -- -- GENERATE OUTPUT -- -- ---------------------------------------------------------------------- --
SELECT DISTINCT travel_advice.journey, type, TIME(time) AS time, -- “Timmeh!” station, platform, train_type, headsign, SUBSTRING(TIMEDIFF(arrival, departure), 1, 5) AS duration FROM travel_advice INNER JOIN ( SELECT journey, COUNT(*) / 2 - 1 AS num_transfers FROM travel_advice GROUP BY journey ) journey_row_count USING (journey) WHERE -- We’re still not interested in alternative suggestions that are -- significantly worse than the best option arrival <= DATE_ADD(earliest_arrival_time, INTERVAL 30 MINUTE) ORDER BY arrival, -- Arrive as early as possible num_transfers, -- Fewer transfers are better departure DESC, -- Depart as late as possible journey; -- Group departures and arrivals for each journey
DROP TEMPORARY TABLE IF EXISTS travel_advice;END $$
DELIMITER ;We can call the PLAN_JOURNEY procedure as follows:
CALL PLAN_JOURNEY( 'Nijmegen Heyendaal', 'Amsterdam Science Park', '2023-06-01 08:00:00');This gives us the following results:
| journey | type | time | station | platform | train_type | headsign | duration |
|---|---|---|---|---|---|---|---|
| 8eee2c55 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:46 |
| 8eee2c55 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:46 |
| 8eee2c55 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:46 |
| 8eee2c55 | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 01:46 |
| 8eee2c55 | Departure | 09:41:00 | Amsterdam Centraal | 13a | NS Sprinter | Amersfoort Vathorst | 01:46 |
| 8eee2c55 | Arrival | 09:49:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 01:46 |
| 598f1521 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:46 |
| 598f1521 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:46 |
| 598f1521 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:46 |
| 598f1521 | Arrival | 09:27:00 | Amsterdam Amstel | 1 | NS Intercity | Den Helder | 01:46 |
| 598f1521 | Departure | 09:32:00 | Amsterdam Amstel | 1 | NS Sprinter | Uitgeest | 01:46 |
| 598f1521 | Arrival | 09:35:00 | Amsterdam Muiderpoort | 8 | NS Sprinter | Uitgeest | 01:46 |
| 598f1521 | Departure | 09:46:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 01:46 |
| 598f1521 | Arrival | 09:49:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 01:46 |
| ab329c3d | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:58 |
| ab329c3d | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:58 |
| ab329c3d | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:58 |
| ab329c3d | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 01:58 |
| ab329c3d | Departure | 09:53:00 | Amsterdam Centraal | 10a | NS Sprinter | Zwolle | 01:58 |
| ab329c3d | Arrival | 10:01:00 | Amsterdam Science Park | 1 | NS Sprinter | Zwolle | 01:58 |
| 1fc35e08 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:58 |
| 1fc35e08 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:58 |
| 1fc35e08 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:58 |
| 1fc35e08 | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 01:58 |
| 1fc35e08 | Departure | 09:39:00 | Amsterdam Centraal | 5b | NS Sprinter | Utrecht Centraal | 01:58 |
| 1fc35e08 | Arrival | 09:44:00 | Amsterdam Muiderpoort | 9 | NS Sprinter | Utrecht Centraal | 01:58 |
| 1fc35e08 | Departure | 09:58:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Zwolle | 01:58 |
| 1fc35e08 | Arrival | 10:01:00 | Amsterdam Science Park | 1 | NS Sprinter | Zwolle | 01:58 |
| 45e9a348 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:58 |
| 45e9a348 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:58 |
| 45e9a348 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:58 |
| 45e9a348 | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 01:58 |
| 45e9a348 | Departure | 09:49:00 | Amsterdam Centraal | 5 | NS Sprinter | Rotterdam Centraal | 01:58 |
| 45e9a348 | Arrival | 09:54:00 | Amsterdam Muiderpoort | 9 | NS Sprinter | Rotterdam Centraal | 01:58 |
| 45e9a348 | Departure | 09:58:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Zwolle | 01:58 |
| 45e9a348 | Arrival | 10:01:00 | Amsterdam Science Park | 1 | NS Sprinter | Zwolle | 01:58 |
| c0369efc | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:58 |
| c0369efc | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:58 |
| c0369efc | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:58 |
| c0369efc | Arrival | 09:27:00 | Amsterdam Amstel | 1 | NS Intercity | Den Helder | 01:58 |
| c0369efc | Departure | 09:32:00 | Amsterdam Amstel | 1 | NS Sprinter | Uitgeest | 01:58 |
| c0369efc | Arrival | 09:35:00 | Amsterdam Muiderpoort | 8 | NS Sprinter | Uitgeest | 01:58 |
| c0369efc | Departure | 09:58:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Zwolle | 01:58 |
| c0369efc | Arrival | 10:01:00 | Amsterdam Science Park | 1 | NS Sprinter | Zwolle | 01:58 |
| f5ed9dac | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 01:58 |
| f5ed9dac | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 01:58 |
| f5ed9dac | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 01:58 |
| f5ed9dac | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 01:58 |
| f5ed9dac | Departure | 09:41:00 | Amsterdam Centraal | 13a | NS Sprinter | Amersfoort Vathorst | 01:58 |
| f5ed9dac | Arrival | 09:46:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 01:58 |
| f5ed9dac | Departure | 09:58:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Zwolle | 01:58 |
| f5ed9dac | Arrival | 10:01:00 | Amsterdam Science Park | 1 | NS Sprinter | Zwolle | 01:58 |
| 0ece0c64 | Departure | 08:18:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 02:01 |
| 0ece0c64 | Arrival | 08:22:00 | Nijmegen | 1b | Arriva Stoptrein | Nijmegen | 02:01 |
| 0ece0c64 | Departure | 08:43:00 | Nijmegen | 3 | NS Intercity | Den Helder | 02:01 |
| 0ece0c64 | Arrival | 10:05:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 02:01 |
| 0ece0c64 | Departure | 10:11:00 | Amsterdam Centraal | 13a | NS Sprinter | Amersfoort Vathorst | 02:01 |
| 0ece0c64 | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 02:01 |
| 33b7969d | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 02:16 |
| 33b7969d | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 02:16 |
| 33b7969d | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 02:16 |
| 33b7969d | Arrival | 09:45:00 | Amsterdam Sloterdijk | 3 | NS Intercity | Den Helder | 02:16 |
| 33b7969d | Departure | 09:53:00 | Amsterdam Sloterdijk | 12 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| 33b7969d | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| ad5ea63c | Departure | 08:18:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 02:01 |
| ad5ea63c | Arrival | 08:22:00 | Nijmegen | 1b | Arriva Stoptrein | Nijmegen | 02:01 |
| ad5ea63c | Departure | 08:43:00 | Nijmegen | 3 | NS Intercity | Den Helder | 02:01 |
| ad5ea63c | Arrival | 09:57:00 | Amsterdam Amstel | 1 | NS Intercity | Den Helder | 02:01 |
| ad5ea63c | Departure | 10:02:00 | Amsterdam Amstel | 1 | NS Sprinter | Uitgeest | 02:01 |
| ad5ea63c | Arrival | 10:05:00 | Amsterdam Muiderpoort | 8 | NS Sprinter | Uitgeest | 02:01 |
| ad5ea63c | Departure | 10:16:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 02:01 |
| ad5ea63c | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 02:01 |
| 267d6590 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 02:16 |
| 267d6590 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 02:16 |
| 267d6590 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 02:16 |
| 267d6590 | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 02:16 |
| 267d6590 | Departure | 09:41:00 | Amsterdam Centraal | 13a | NS Sprinter | Amersfoort Vathorst | 02:16 |
| 267d6590 | Arrival | 09:46:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| 267d6590 | Departure | 10:16:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| 267d6590 | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| 7354f15a | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 02:16 |
| 7354f15a | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 02:16 |
| 7354f15a | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 02:16 |
| 7354f15a | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 02:16 |
| 7354f15a | Departure | 09:53:00 | Amsterdam Centraal | 10a | NS Sprinter | Zwolle | 02:16 |
| 7354f15a | Arrival | 09:58:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Zwolle | 02:16 |
| 7354f15a | Departure | 10:16:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| 7354f15a | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| b89059e7 | Departure | 08:03:00 | Nijmegen Heyendaal | 2 | Arriva Stoptrein | Nijmegen | 02:16 |
| b89059e7 | Arrival | 08:07:00 | Nijmegen | 35 | Arriva Stoptrein | Nijmegen | 02:16 |
| b89059e7 | Departure | 08:13:00 | Nijmegen | 3 | NS Intercity | Den Helder | 02:16 |
| b89059e7 | Arrival | 09:35:00 | Amsterdam Centraal | 8 | NS Intercity | Den Helder | 02:16 |
| b89059e7 | Departure | 09:49:00 | Amsterdam Centraal | 5 | NS Sprinter | Rotterdam Centraal | 02:16 |
| b89059e7 | Arrival | 09:54:00 | Amsterdam Muiderpoort | 9 | NS Sprinter | Rotterdam Centraal | 02:16 |
| b89059e7 | Departure | 10:16:00 | Amsterdam Muiderpoort | 2 | NS Sprinter | Amersfoort Vathorst | 02:16 |
| b89059e7 | Arrival | 10:19:00 | Amsterdam Science Park | 1 | NS Sprinter | Amersfoort Vathorst | 02:16 |
That’s it, we’re almost done! There’s just one more thing we need to do, but I’ll leave that one for the next post…