DBMerge: A database-agnostic Python UPSERT module to simplify ETL pipelines

Hi everyone, I’d like to share an open-source library I’ve been developing for the Python community: DBMerge. Consider it like an advanced version of pd.to_sql. It is designed to simplify common but tedious task of syncing data to a SQL database. Instead of writing custom MERGE, UPSERT or ON CONFLICT queries for different SQL dialects, DBMerge performs INSERT, UPDATE, and DELETE operations automatically in a single step. Because it is built on top of SQLAlchemy Core, it is fully database-agnostic. It has currently been thoroughly tested with PostgreSQL, MySQL/MariaDB, SQLite and MS SQL Server. How It Works Under the Hood The underlying logic focuses on performance and reliability: Staging: The module first creates a temporary staging table in the database and loads your entire incoming dataset into it using a fast bulk INSERT. Reconciliation: It then executes optimized UPDATE, INSERT, and DELETE statements against the target table by comparing the target table with the temporary one. (