Lanes as Vectors — Autobahn Autopilot

After a successful lane detection, it is time to extract valuable vectors from tensors.

In case you missed part 1: Autobahn Autopilot — Euro Truck Simulator 2

This frame is directly taken from the game and size is set to the model’s size which is 256x128.

Here is the result after lane detection (part 1) ran on the frame.

This part shows how to filter important lines and find them as vectors.

Let’s remember what is in one of tensors.

> Grab a frame
// check part 1

> Detect lanes
// check part 1

tensor[0][0]

Let’s take a closer look.

This is a 256x128 binary matrix, and elements are “1” only if they are detected as a piece of a lane.

A line is always displayed as “1”s and ends with “0” from all sides.

Labeling can separate those lines from each other.
Keyword is image processing.

from scipy.ndimage.measurements import label

Old documentation, New documentation

Get the prediction as numpy array to process.

data = pred[0][0].cpu().numpy()

This method helps you to label all neighbor “1”s.

labeled_array, num_features = label(data)

num_features tells you how many labels are created — how many different possible lines are detected.
It will not going to look as you expected until you request with a proper structure.

Default structure is;

[[0,1,0],
 [1,1,1],
 [0,1,0]]

However, we are looking for all neighbors.
For this use case, a different structure is needed;

[[1,1,1],
 [1,1,1],
 [1,1,1]]

So the correction;

labeled_array, num_features = label(
    data, structure=[[1, 1, 1], [1, 1, 1], [1, 1, 1]]
)

Result;

Closer look;

All “1” are labeled with a different number as long as the section can continue as our transform value.

Find Vectors

In this matrix, with the following actions, vectors can be extracted.

1. Look for labels
   numpy.argwhere
2. Convex hull
   spatial.ConvexHull
3. Get distance between pairs
   spatial.distance_matrix
4. Select furthest candidates
   numpy.unravel_index

A quick look into methods;

numpy.argwhere

argwhere finds the indices of array elements that are non-zero, grouped by element.

spatial.ConvexHull

In geometry, the convex hull or convex envelope or convex closure of a shape is the smallest convex set that contains it.

spatial.distance_matrix

Computes the distance matrix and returns the matrix of all pair-wise distances.

numpy.unravel_index

unravel_index converts a flat index or array of flat indices into a tuple of coordinate arrays.

Source

vectors = []
for i in range(num_features):
    index = i + 1
    # look for the next group label
    indices = np.argwhere(labeled_array == index)
    if len(indices) < THRESHOLD:
          # not an important line if under threshold (optional)
          continue
     # convex hull
     candidates = indices[spatial.ConvexHull(indices).vertices]
     # distance between pairs
     dist_mat = spatial.distance_matrix(candidates, candidates)
     # furthest candidates
     points = np.unravel_index(dist_mat.argmax(), dist_mat.shape)
     vectors.append([
         candidates[points[0]][1],
         candidates[points[0]][0],
         candidates[points[1]][1],
         candidates[points[1]][0],])

lotdcotw/ets2ap

Now, you have a list of vectors which you can precisely work with.

Tune in for part 3; steering better with precise vectors!

Follow to see more;

Linkedin: https://www.linkedin.com/in/lotdcotw
Youtube: https://www.youtube.com/channel/UCWh19xX-0WBvzrlEG5gYh-Q
Twitter: https://twitter.com/lotdcotw
Instagram: https://instagram.com/lotdcotw
Facebook: https://www.facebook.com/lotdcotw