This is the classifier backend for https://github.com/LiteralGenie/jobber-site
Skill labels are currently generated via regex. All other labels are generated with structured output from an LLM.
Install dependencies
git clone [email protected]:LiteralGenie/jobber-classifier.git
cd ./jobber-classifier
python3 -m venv venv
. ./venv/bin/activate
python -m pip install -r requirements.txt(optional) Configure llama-cpp-python to use the GPU
Remember to also set use_gpu = true in the config file described in the next step.
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 python -m pip install --force-reinstall --upgrade --no-cache-dir llama-cpp-python
Create a config file using the example template.
cp ./src/config/config_example.toml ./src/config/config.toml
nano ./src/config/config.toml
Add the skills / duties from the config to the SQLite database.
(At least one skill and one duty need to be defined.)
python ./src/bin/apply_config.py
Populate the posts table in the database with job data (left as an exercise for the reader).
Or alternatively download a copy of the current Jobber database.
curl https://jobber.velchees.dev/api/export/sqlite
mv db.sqlite /path/in/config/db.sqlite
Generate labels for each post.
python ./src/bin/run_extract.py