Motivation

My two Android apps Metal Only and MensaUPB require some kind of online service to get their data from and - since I cannot force users to update apps easily - I wrote my own API as a wrapper to the upstream APIs provided by https://www.metal-only.de/ and https://studierendenwerk-pb.de/. That way, whenever one of these pages changes their API, I can just adapt the wrapper and keep my own API backwards-compatible so the apps themselves don’t require any updates.

The first post in this series was about the original problem: there is a data-set of strings with potential duplicates and many typos. The second was about computing the similarity of two strings. In this post we will focus on the question

Can we group multiple strings of a set by similarity?

**Matching the names: **The first step is to match the names. I hope, that the following code is documented good enough.

Cleaning some data with Clojure 1 provides some context on this project. This post here focusses on the question

“Can we tell, if two strings are similar?”.

Levenshtein Distance: My first idea was to use the Levenshtein Distance. This metric counts the minimum number of operations needed to transform one string into the other when allowing insertion, deletion or substition of characters. Its definition is as follows:

Context: The students’ council mathematics and computer science in Paderborn (FSMI) provides a service called “Klausurausleihe”. Students may come to the FSMI and lend exams and protocols to prepare themselves  for exams.

To manage the dataset behind this services, the so-called klausurtool is used. The FSMI an manage the set of exams available and their status. Exams are represented as tuples (examiner, date, content, lendingStatus, lecture) [0].

The problem: For every single exam, the examiner (and other data has to be entered). There were no rules on how to enter the examiner. Also, the klausurtool did not use UTF-8.