XiaokeAILabsOPEN EXPERIMENTS · UPDATED ONGOINGLYUnderstand AI engineering
Understand AI engineering
through experiments.
From model training and retrieval to inference deployment, we use code and measurements to ask what works, under which conditions, and how to test it on your own data and hardware.
Start with a question
Explore methods alongside their data, settings, and results.
Training and fine-tuning
Prepare data, adapt embedding and reranking models, and assess them with retrieval metrics.Inference and performance
Compare quantization, serving engines, and concurrency; estimate memory and bandwidth for deployment.Retrieval and storage
Explore multi-stage retrieval and compare vector database recall, latency, and storage costs.Documents and multimodal AI
Study OCR, image fine-tuning, and cross-modal deployment across devices and their limitations.What measurements can tell us
Every result depends on its data, hardware, and method.
Check outputs as well as speed
On four OCR samples on GB10, total model generation time improved by 2.36× with identical greedy outputs. Timing excludes image loading and preprocessing; the result does not represent all documents.
Explore OCR acceleration →Build your own comparison
Models, data, and hardware affect results. Experiment records include configurations and methods to help you test your own workload.
Browse experiments →Experiment index
Choose a topic to inspect methods, code, results, and their limits. English guides are linked where available; other entries open in Chinese.
Training, fine-tuning, and retrieval
Chinese embedding fine-tuningLearn in-batch contrastive training with query-document pairs; the four built-in examples do not establish a retrieval improvement.
Distillation, multimodal AI, and fine-tuning
Embedding distillationOn fixed candidates for 300 held-out queries, distillation nDCG@10 was 0.0057 higher than the base, below the preset 0.02 target.
ColBERT distillationA real-teacher rerun found a small difference from supervision; one seed does not establish an advantage.
Visual multi-vector document retrievalA page-retrieval baseline and synthetic-page training comparisons; this run showed no distillation gain.
Multimodal embedding distillationCompare paired supervision and teacher distillation; on 100 test queries, R@1 was 0.91 for distillation and 0.93 for supervision.
GUI grounding fine-tuningOn 48 stratified held-out samples, the base hit 34 targets and LoRA hit 42; this does not establish cross-application generalization.
Video generation trained on teacher samplesCompare teacher-generated and student-generated training data. A small sampled-frame review did not show the teacher-data group outperforming the base.
Image generation with synthetic training dataFine-tune a student on teacher-generated data. A camera teacher-advantage gate failed; a mug development evidence set is available without weights; it is not a final test or proof of successful distillation.
Identity image fine-tuningTrain an identity LoRA; the historical base and adapted outputs use different resolutions and are not directly comparable.
Character video fine-tuning and diagnosticsExamine base generation, seeds, and LoRA outputs; a small held-out evaluation does not establish that the target character was learned.
Inference, deployment, and costs
Language JEPA trainingIn the toy arithmetic test, 21 of 40 problems match training problems with reversed operands; no improvement established.
Diffusion language model speedCompare generation paths on three prompts with unequal output lengths and unmeasured quality.
GB10 inference slotsCompare throughput and completion times for one versus eight slots on the same model and engine; two runs per setting.
Language model quantizationCompare model size, perplexity, and CPU speed; the BF16 path performed unusually slowly.
Inference cost modelsMemory examples and theoretical models; performance estimates have not been independently validated.
Data structure costsExplore teaching models of indexes and storage; historical format records cannot support selection.
Document understanding and multimodal AI
Portrait mattingEvaluate subject regions, boundaries, and forward time on 30 still images; results do not establish video performance.
Table structure recognitionExamine cropping and structure predictions on one synthetic table; text recognition is out of scope.
Text detection and recognitionPass detected text regions to a recognizer; the two-stage result covers one synthetic table.
