AI Agents for Magazine Article Processing

Agent Pipelines and MLOps for Magazine Summarization, Translation, Classification and Conversion

This is a company project, so this page covers only what can be shared: the goals, the system design, my contributions and the results.

I built and run an LLM + VLM system that processes magazine articles end to end. It summarizes, translates, classifies and compresses article text, cleans up OCR output, and rebuilds scanned magazine pages into web-view articles ready to publish. The same agents answer on-demand requests through FastAPI endpoints and process whole batches as Dagster jobs.

Period
Mar 2025 – present
Role
Designed and built the agents, serving, orchestration, evaluation and MLOps
Scale
24K articles summarized · 520K articles classified · 4.3K articles delivered per round
Models
Local LLM on vLLM · OCR and a layout VLM for scanned pages · embedding model

The three case studies show where I measured a bottleneck in this system and what changed once it was fixed.

System overview

Every feature is an agent in one library that knows nothing about the web. FastAPI wraps that library for on-demand calls from the text viewer and bulk jobs. Dagster wraps the same code for batch work, where every article is its own run. All models are served locally: an LLM on vLLM for text, and OCR plus a layout VLM for scanned pages.

System architectureArticle text and scanned pages enter through FastAPI endpoints or Dagster jobs. Both call the same agent library: summarize, translate, classify, compress, OCR cleanup and web-view conversion. The agents run on locally served models: an LLM on vLLM, OCR, a layout VLM and an embedding model. Alongside: Jenkins deployment, Langfuse prompts and traces, DeepEval for LLM as Judge evaluation, Prometheus monitoring.INPUTArticle text · scanned magazine pagesON DEMAND · HTTPFastAPI endpointsone endpoint per feature · 16 slotsBATCH · ONE RUN PER ARTICLEDagster jobsstatus, failures and retries per articleAGENT LIBRARY · LANGGRAPHSUMMARIZESummaryany lengthTRANSLATETranslate4 agentsCLASSIFYTaggingFAISS → LLMCOMPRESSTo 80%per chunkOCR REFINEOCR fixesline breaksWEB VIEWWeb viewfrom scansMODELS · SERVED LOCALLY · 2× L40SLLM on vLLM · OCR · layout VLM · embeddingsThe server swaps the vLLM stack per mode · OCR runs on a remote GPU host over gRPCDEPLOYJenkinsPROMPTS · TRACESLangfusefetched on every callEVALUATIONLLM as Judgemeasured with DeepEvalper-articleMONITORINGPrometheusGrafana · LokiOpenAI-compatible API

Two L40S GPUs can't hold every model at once, so the server swaps the vLLM stack for default, summarization or classification mode. Jenkins deploys the service, waits until it is ready and runs smoke tests.

LangChain · LangGraph · Langfuse · vLLM · FastAPI · Dagster · DSPy · deepeval · Prometheus · Grafana · PostgreSQL · Docker


Case studies

Case 01 · MLOpsMake the batch job visible, then restructure itThe vLLM dashboard showed a prefix-cache hit rate of only 24.7%. Parallel processing and restructured prompts raised it to 45.7%, cutting prefill compute 28% and prefill time 10%. Moving each agent call into its own Dagster step made failures visible and cut recovery tokens 37%.#vllm#monitoring#dagster#recovery Case 02 · Dual-teacher distillationFrom an LLM tagging agent to a 2 ms confidence-aware classifierI distilled a structured-output LLM tagger into a confidence-aware encoder classifier. A second LLM teacher, a judge, then corrected the labels the first one's pipeline got wrong. 3,244 ms → 2 ms per article; judge-based F1 0.614 → 0.801.#distillation#dual-teacher#llm-judge#confidence Case 03 · Self-RefineSummarizing special-format articles — silent defects 16.9% → 10.0%Per-format rule cards (ExpeL), partial rewrites driven by the Judge's critique (Reflexion) and two gates that hand articles to a human. A blind evaluation met all three pre-registered conditions.#self-refine#expel#reflexion#dspy