Tachibana 2 — Code Reasoning with DeepSeek R1
Corpus of complex prompts to assess reasoning skills in programming the DeepSeek R1 (685B) model.
Description
Tachibana2-DeepSeek-R1 is a corpus of more than 27,000 prompts generated synthetically to test the programming reasoning skills of the DeepSeek R1 model. Prompts come from an enriched variant of the original Tachibana dataset, and responses are produced automatically by the R1 model without human filtering. The aim is to assess the quality, consistency and logic of the responses generated by an LLM specializing in code.
What is this dataset for?
- Evaluate an LLM's ability to reason around programming logic
- Test the robustness and the possible errors generated by very large models
- Create benchmarks or refine specialized models using a coding assistant
Can it be enriched or improved?
Yes. Although this dataset is provided “as is”, it can be enriched by human annotations (validation, correction, error classification). It can also be used as a basis for tighter filtering or targeted pre-processing before fine-tuning. An automatic response scoring system could also be integrated.
🔎 In summary
🧠 Recommended for
- Specialized LLM developers
- Reasoning AI researchers
- AI programming tutorial projects
🔧 Compatible tools
- VLLM
- Code Llama
- DeepSeek
- Hugging Face Transformers
- LangChain
💡 Tip
To take advantage of the dataset, filter examples with infinite loops or inconsistent outputs before fine-tuning.
Frequently Asked Questions
Is this dataset suitable for the evaluation of general LLMs?
Yes, but it's mostly relevant for testing specific skill in code reasoning.
Are the answers validated manually?
No, the outputs of the R1 model are provided as is, without correction or annotation.
Can it be used for supervised training?
Yes, but pre-processing is recommended to filter out incorrect or outlier answers.
