~postgresweekly | Issue 553 - pgvector gets even better (8)

Squawk: A Linter for Postgres Migrations — Prevent unexpected downtime caused by database migrations and encourage best practices around schemas and SQL.

Crunchy Bridge for Analytics: Your Data Lake in Postgres? — Marco ponders: “What if you could use PostgreSQL to easily query your data lake with state-of-the-art analytics performance?” Crunchy Bridge for Analytics is a new part of Crunchy Data’s Postgres platform offering S3 integration, CSV/Parquet querying support, and bidirectional data lake ↔ Postgres import/export.

The 150x pgvector Speedup: A Year in Review — pgvector provides Postgres with extra data types, indexing, and querying facilities to work with high dimensional vectors (such as are common with LLMs and embeddings). The past year has seen a huge leap in its capabilities and performance.

Postgres Bloat Minimization — Tables occupying more space than expected is a common cause of complaint, but there are a variety of strategies you can employ so that space is used more efficiently.

Papa's Got a Brand New RAG — RAG (Retrieval-Augmented Generation) is a technique used alongside large language models where you fetch data to include in prompts to improve the responses LLMs give. Shaun demonstrates bringing all the parts together to build a simple RAG-based query system in Python with pg_vectorize doing the heavy lifting on the Postgres side.

PeerDB: A Simple, Fast Way to Replicate Postgres Data Elsewhere — A data replication system built by former Microsoft and Citus Data employees that targets use cases where you need to stream data from Postgres to data warehouses, queues, and other storage engines. This week’s v0.13.0 release introduces an Elasticsearch connector. GitHub repo.

pg_tier: Tier Data to External Amazon S3 Storage — Once data becomes older or much less frequently accessed, it might be more cost effective to store it elsewhere. pg_tier makes it easier to push a table over to S3 while still maintaining the ability to query it (at the price of some added latency, naturally).

Improve Postgres Performance and Reduce Costs with OtterTune — Automatically tune and optimize knobs, plus get index and query recommendations, along with table and autovacuum insights. Customers have improved performance over 90% and saved thousands of dollars in database labor costs. Do a free, 30-day trial.