Wirewalk Instituteof AI
bld-215 Intermediate Waitlist

Embeddings and Vector Databases

For engineers who need semantic search, deduplication, clustering or recommendation at scale: choose an embedding model, size and tune a vector index, and run a vector database in production.

The curriculum is being prepared and reviewed. Enrollment is not open yet.

About this course

Embeddings turn text, images and other data into vectors whose distances mean something; vector databases make finding the nearest ones fast. This course treats both as engineering components with measurable trade-offs. You will learn how contrastive training shapes an embedding space, how to read MTEB results without over-trusting them, how to evaluate candidate models on your own data, and how to cut storage with Matryoshka dimensions and scalar or binary quantization. Hosted models from OpenAI, Google (including the multimodal Gemini Embedding 2), Cohere and Voyage AI are compared with open models run through Sentence Transformers.

The second half is about the database: exact versus approximate search, HNSW and IVF-PQ parameters, disk-based indexes such as DiskANN, metadata filtering and why it breaks recall, sparse and multi-vector representations, and a structured comparison of pgvector, Pinecone, Qdrant, Weaviate, Milvus and LanceDB. It closes with operations: capacity and memory sizing, recall monitoring, multi-tenant isolation and migrating to a new embedding model without downtime. RAG pipeline design is covered in bld-210; this course goes deeper on the vector layer itself. The course ends in an automated exam; passing earns a Wirewalk Institute of AI certificate of completion and assessment.

What you will be able to do

  • Explain how contrastive training, pooling and normalization determine what an embedding space captures and which distance metric to use
  • Evaluate candidate embedding models on a domain test set and justify a choice beyond leaderboard rank
  • Reduce vector storage with Matryoshka truncation and scalar, product or binary quantization while measuring the recall cost
  • Configure HNSW and IVF-PQ indexes and tune their parameters against a recall, latency and memory budget
  • Design filtered, hybrid and multi-tenant queries that keep recall acceptable
  • Compare pgvector, Pinecone, Qdrant, Weaviate, Milvus and LanceDB on deployment model, filtering, scale and cost for a given workload
  • Plan and execute a zero-downtime migration to a new embedding model, including dual-writing and validation

Syllabus

6 modules · 20 lessons · about 13 hours

Module 1: How Embedding Models Work

From contrastive training to the geometry you query against.
3 lessons
  1. Contrastive Training, Pooling and What a Vector Actually Encodes
  2. Cosine, Dot Product and Euclidean: Normalization and Metric Choice
  3. Beyond Text: Image, Code and Multimodal Embedding Spaces

Module 2: Choosing and Evaluating Embedding Models

Hosted and open models, benchmarks, and testing on your own data.
4 lessons
  1. Reading MTEB and MMTEB Without Being Misled
  2. Hosted Models Compared: OpenAI, Gemini Embedding, Cohere Embed and Voyage
  3. Open Models with Sentence Transformers and Building a Domain Eval Set
  4. Fine-Tuning an Embedding Model on In-Domain Pairs

Module 3: Compressing Vectors

Fewer dimensions and fewer bits without losing the results that matter.
2 lessons
  1. Matryoshka Representations and Dimension Truncation
  2. Scalar, Product and Binary Quantization with Rescoring

Module 4: Index Internals

How approximate nearest-neighbor search trades recall for speed and memory.
4 lessons
  1. Exact Search with FAISS and When Brute Force Is Enough
  2. HNSW Parameters: M, efConstruction and efSearch Measured
  3. IVF-PQ, DiskANN and Indexes That Live on SSD
  4. Filtered Search: Pre-Filter, Post-Filter and Filter-Aware Graphs

Module 5: Vector Databases Compared

Extensions, managed services and open-source engines for different workloads.
3 lessons
  1. pgvector and pgvectorscale: Vectors Inside PostgreSQL
  2. Pinecone, Qdrant, Weaviate, Milvus and LanceDB: A Structured Comparison
  3. Sparse Vectors, Multi-Vector Late Interaction and Native Hybrid Queries

Module 6: Operating Vectors in Production

Sizing, monitoring, isolating tenants and changing models safely.
4 lessons
  1. Capacity Planning: Memory, Disk and QPS Sizing Worksheets
  2. Recall Monitoring and Index Rebuild Strategy
  3. Multi-Tenancy Patterns: Namespaces, Partitions and Per-Tenant Indexes
  4. Re-Embedding Migrations: Dual-Write, Backfill and Cutover

Exam and certificate

The exam unlocks once you have completed every lesson. It draws 20 questions from a larger bank, spread across the modules, with the answer options shuffled for each attempt. You have 40 minutes and need 80% to pass. After a failed attempt you can try again after 24 hours, up to 3 attempts in total. Feedback is given per module, never as the answer key.

Passing earns a certificate of completion and assessment from the Wirewalk Institute of AI with a unique ID that anyone can check at institute.wirewalk.com/verify. It is not an accredited qualification or a vendor certification. How certification works.

The Wirewalk Institute of AI is independent. This course teaches publicly documented products and is not affiliated with or endorsed by OpenAI, Google, Cohere, Voyage AI, Hugging Face, PostgreSQL pgvector, Pinecone, Qdrant, Weaviate, Milvus, LanceDB, or FAISS. Product and company names are trademarks of their respective owners.

More in Building AI Applications

All 14 courses in this track
bld-101 Basic Waitlist

No-Code AI Assistants: Custom GPTs, Projects, Gems and Copilot Studio

A hands-on course on building no-code assistants with the builders already in your workplace tools: ChatGPT projects, custom GPTs and their successor plugins, Claude Projects and Skills, Gemini Gems, and Microsoft 365 Copilot Agent Builder and Copilot Studio. You leave with a tested, documented assistant and a plan to share it safely.

7 h · 17 lessons Join the waitlist
bld-201 Intermediate Available

Building with LLM APIs Across Vendors

A hands-on, code-first course on the three major LLM APIs side by side: messages, streaming, structured outputs, tool calling, vision, cost, caching, batch, retries, abstraction layers, cloud routes and testing. You finish by building and hardening a small multi-vendor service.

14 h · 18 lessons See price at checkout
bld-210 Intermediate Available

Retrieval-Augmented Generation Systems

A practitioner's course on retrieval-augmented generation: when to use it, how to ingest, chunk, embed and index documents, how to combine BM25 and vector search with reranking, and how to measure, debug and operate the result. Every module includes working Python sketches.

14 h · 16 lessons See price at checkout