33 lines
3.2 KiB
Plaintext
33 lines
3.2 KiB
Plaintext
|
|
|
|
|
|
# Context for the project
|
|
|
|
I have a reMarkable Paper Pro for notes taking. I want to have a suite of scripts to sync the files from the reMarkable tablet to my computer, use GenAI endpoints to OCR my handwritten pages to text in a structured way and then sync the data to my Obsedian markdown vault.
|
|
I mostly take meeting notes, including date, name of the meeting, sometimes name of participats and the content. On the content its information / update notes / talking points, but also TODOs (both for me as well for others).
|
|
I mostly use lists (structured by hyphens, dots or stars)
|
|
Todos will be marked with "TODO" and name of the person (or abbrevation like CWE for Christian Wesp).
|
|
In the end, I want to use this tool suite to transcripte my notes, summarize them and extract key information and todos from the notes.
|
|
|
|
# Execution environment:
|
|
* The system environment is Linux / Ubuntu
|
|
* scripts should be written in Bash and especially Python
|
|
* AI endpoints will be Azure OpenAI endpoints or optionally the public OpenAI endpoints (configurable)
|
|
|
|
# Different steps:
|
|
|
|
* First script: sync files from the reMarkable tablet to a local folder (via rsync, see01 Sync Docs.bash)
|
|
* Second script: A script which parses all the metadata files (.metadata, .content) with the actual data files (.rm files in subfolders). See the test script "0a test.ipynb" for an overview how the files are connected via json metadata. This script should create a small local database, which holds:
|
|
** list / table how documents are strucutred, name, folder, latest change date
|
|
** an algorithm on which pages are new, have been altered or changed (e.g. on timestamps in the metadata and with cryptographic hashes on the .rm files)
|
|
** output of .rm files which should be processed (new pages / changed pages)
|
|
|
|
* Third script: taking the input from the first script, process each new / updated page and convert it to a image file which can be processed with an OCR / AI engine. For example, use https://github.com/ricklupton/rmc to convert .rm files to svg, and then magik to convert svg to png.
|
|
|
|
* 4. script: send each image to an AI endpoint to OCR it. use configurate templates for the prompts so I can adjust the system prompt, give instruction how the AI shold format but also give additional context. Create initial proposals for prompts. The data is then saved into markdowns. On this, please think about and make a judgment call:
|
|
** Beside the transcription, I also want to extract key take aways, todos and summaries, which I can later link in knowledge graphs. Decide if this should be done in the same script as the OCR / AI OCR script, or in a seperate script
|
|
** also keep in mind, that each page from reMarkable is a seperate image, and the seperate outputs need to be stiched together into a single document.
|
|
** Also, often pages have notes across different meetings on the same day. I will always create a new page for a new date, and each date will have several pages. Keep all notes for a single date in a single markdown - and probably create / extract cross-date information in a seperate file. But please come up with a good recommendations.
|
|
|
|
Please check, if I miss something or if you would extend the scope of this project
|