CSV-to-SQL DDL & Bulk Insert Generator

Relational Database Schemas & DDL

The structural layout of a relational database is defined using **Data Definition Language (DDL)** commands—specifically the `CREATE TABLE` statement. When converting raw CSV text files into databases (such as PostgreSQL, MySQL, or SQL Server), mapping columns to appropriate mathematical, date, or character types is crucial to protect storage efficiency and maintain data integrity.

Storing every column as a generic `VARCHAR` text field degrades index performance, slows join lookups, and breaks standard relational functions (such as sorting dates chronologically or performing math averages on decimal price points).

Standard Database Normalization Guidelines

Relational systems rely on Normalization—the mathematical process of organizing columns to reduce redundancy and eliminate update anomalies. The core stages include:

• 1st Normal Form (1NF): All table attributes must hold atomic, single-valued cells. No arrays or lists.
• 2nd Normal Form (2NF): Must be in 1NF and all non-key columns must fully depend on the table's primary identifier.
• 3rd Normal Form (3NF): Must be in 2NF and non-key columns cannot depend transitively on other non-key columns.

The Type Inference Engine Explained

This utility's local type inference engine scans cell values down each vertical column to determine the narrowest, most accurate SQL storage format:

• BOOLEAN / BIT: Deduced when columns contain only binary states (`true`, `false`, `1`, `0`).
• INTEGER: Guessed when all inputs are strictly numeric digits without periods.
• DECIMAL / NUMERIC: Preferred for prices or fractional scores. PostgreSQL and MySQL use exact decimals `DECIMAL(10,2)` to prevent floating-point rounding errors typical of binary float columns.
• TIMESTAMP: Detected when fields adhere to standard ISO date formats, converting them to UTC temporal storage blocks.

Optimizing Bulk INSERT Queries

When pushing thousands of compiled INSERT queries into production clusters, wrapping the package in a **Database Transaction** (`BEGIN;` and `COMMIT;` blocks) reduces overhead by up to 95%. This blocks the database from constantly refreshing physical indexes to disk on a single row level, batching the indexing update at the end of the transaction instead.