8 commits

Author	SHA1	Message	Date
Eduard Urbach	25f6928f40	Simplified tokenizer	2025-02-02 11:11:59 +01:00
Eduard Urbach	ff40760186	Added more tokenizer tests	2025-01-31 11:42:34 +01:00
Eduard Urbach	a466281307	Simplified file structure	2024-08-07 19:39:10 +02:00
Eduard Urbach	722d07c321	Reorganized file structure	2024-06-10 15:51:39 +02:00
Eduard Urbach	18993e0855	Added token tests	2024-06-06 21:35:14 +02:00
Eduard Urbach	c0f399df7f	Improved file structure	2024-06-05 15:51:05 +02:00
Eduard Urbach	8de28852cd	Improved tokenizer	2023-10-31 21:13:14 +01:00
Eduard Urbach	21cf4b5c6d	Added a tokenizer	2023-10-31 11:57:37 +01:00