A Prompt Is Not All You Need: Building a LLM Synthetic Data Pipeline with Open-Source Tools

Today, as the open-source model ecosystem flourishes, developers can easily access top-tier large language models (LLMs) like Gemma4 and Qwen3.5. However, implementing these technologies in highly regulated fields such as finance and healthcare often faces a significant obstacle: data privacy. Due to restrictions on personally identifiable information (PII) and regulations, teams cannot access real business data for PoC, performance validation, or RAG system stress testing. Without usable data for validation, even the best open-source models struggle to integrate into real-world scenarios.

To overcome this "data accessibility" challenge, synthetic data emerges as a crucial solution.

Many teams initially attempt manual prompt engineering to generate test data. However, when datasets reach thousands of records with diverse requirements, boundary conditions, and logical consistency, this method quickly encounters quality control, validation, and scalability issues.

This session will feature Nvidia's open-source tool Data Designer, demonstrating a reproducible pipeline that includes synthetic PII/domain QA seed schema, DAG configuration, validator design, LLM-as-a-Judge rubric, and data quality inspection reports. All sample data and code will be released under open-source licenses, enabling participants to adapt them directly for their own RAG testing or PoC datasets.

Through this practical case study, we aim to help open-source developers and enterprise IT teams generate high-quality test datasets effortlessly using open-source tools, strengthening the final stage of AI implementation.