PMC OA Markdown with Embeddings
Scientific corpus from PMC Open Access, enriched with vector representations (embeddings) for each article. Markdown format.
Several thousand articles, 2048D embeddings, Markdown format + JSON vectors
CC (Creative Commons, not specified precisely)
Description
PMC OA Markdown Embeddings is a data set structured from open access scientific articles (PubMed Central). Each article is stored in Markdown format, accompanied by a dense embedding vector (dimension 2048) generated via the model Qwen3-Embedding-4B. This allows for powerful semantic research in large biomedical collections.
What is this dataset for?
- Perform a vector or semantic search in scientific corpora
- Creating a basis for retrieval-augmented generation (RAG) systems
- Train or evaluate biomedical models in summary or question-and-answer tasks
Can it be enriched or improved?
Yes. You can refine embeddings with specialized models (biomed), add metadata (DOI, journal, date), or filter by discipline. Articles can also be combined with their abstracts or associated figures to create multimodal corpora.
🔎 In summary
🧠 Recommended for
- AI health engineers
- Biomedical NLP researchers
- Scientific assistant developers
🔧 Compatible tools
- DO
- Qdrant
- LangChain
- Hugging Face Transformers
💡 Tip
Use cosine similarity between embeddings to create an intelligent search engine without supervision.
Frequently Asked Questions
Does this dataset contain the full texts of the PMC articles?
Yes, each entry contains the full text of a PMC article in Markdown format.
Can embeddings be used with tools like FAISS or Qdrant?
Yes, the 2048D vectors can be directly used for vector research with these tools.
Can this dataset be combined with other biomedical corpora?
Absolutely. It can be cross-referenced with PubMed, BioASQ, or other databases to create rich QA or summary systems.
