ML engineer¶
Where to start¶
Choose a dependency profile in the profile quick start, then review the ML/RAG layer in the repository explanation.
Where the code lives¶
src/codex_ml/: training, evaluation, data, models, inference, serving, and pluginssrc/rag/: retrieval, embedding, indexing, and evaluationtraining/andtokenization/: installed root packages explicitly mapped bypyproject.tomlconfigs/: Hydra-compatible profiles and schemastests/: unit and integration coverage for those packages
What this role cares about¶
Reproducible configuration, offline-safe tests, deterministic evaluation, optional dependency boundaries, checkpoint compatibility, and CPU-friendly validation.
Key technologies¶
PyTorch, Transformers, Datasets, Accelerate, PEFT, scikit-learn, SentenceTransformers, FAISS, ChromaDB, Hydra, OmegaConf, MLflow, and Ray Serve.
Typical workflow¶
- Use
runtimefor inference/RAG work orfullfor training and development. - Select or add configuration under
configs/. - Work inside the relevant
src/package and its focused tests. - Validate locally without requiring model downloads or network access in CI.
- Run the relevant ML tests before broader project validation.
Common gotchas¶
- Base dependencies are always installed; extras add capabilities rather than replacing the base set.
runtimeincludes substantial ML and serving dependencies; it is not a minimal training-only profile.fullis the profile that includes test and quality tooling.- CI-facing ML components must not download models or data unless an explicit, documented opt-in permits network access.