From Punch Cards to AI-Native Databases

The article traces more than a century of the history of data storage and processing — from Hollerith's punch cards used in the 1890 US census, through magnetic drums, tapes, and disks, all the way to today's AI-native databases. It describes key technological milestones including the von Neumann architecture, the founding of IBM, and sequential versus direct data access, and explains why the database market has reached 150 billion dollars with PostgreSQL's dominance.

From Hollerith's punched cards of 1890 to the AI-native databases of early 2026 — how data storage models changed and why PostgreSQL came to dominate a $150 billion market.

Database technology has been transformed over more than a century, from manually punched cards to distributed, AI-native systems serving billions of users. Each generational shift was a response to a specific technological and business limitation of the previous model — from mechanical tabulators through sequential tape processing, rigid hierarchies, declarative SQL, to horizontally scalable NoSQL and back to distributed SQL. In 2025 the database market reached a value of $150 billion and is heading toward $329 billion by 2031, with the dominant forces being the convergence of SQL and NoSQL, the rise of vector databases for AI, and the phenomenal growth of PostgreSQL as the "universal database."

The history of data storage and processing begins with the problem of the American census. The 1880 census took eight years — with a growing population, there was a risk that the results of the 1890 census would not be finished before the next census in 1900. In 1888 the Census Bureau announced a competition for the fastest processing method. Two participants processed the test data in 144.5 and 100.5 hours. The third — Herman Hollerith (1860–1929), a graduate of the Columbia School of Mines and a former Census Bureau employee — captured the data in 72.5 hours and prepared it for tabulation in just 5.5 hours (compared with the competitors' 44.5 and 55.5 hours).

Hollerith's system had three key components. The pantograph (punch) allowed the operator to manually transfer data from the census sheets onto cardboard punched cards — each hole at a precisely defined position represented a specific piece of information (sex, age, race, marital status, citizenship). The tabulator "read" the cards using a matrix of metal pins — when a pin passed through a hole, it contacted a mercury cup beneath the card, thereby closing an electrical circuit. The current activated electromagnets on one of 40 dials, each with a capacity of 0–9,999. A bell signaling the completion of reading a card alerted the operator to record the data. The sorting box automatically opened the correct compartment for filing the card according to the detected combinations of holes.

An experienced operator processed 80 cards per minute. In total, approximately 60 million cards were punched for the 1890 census. The result: a rough population count was obtained in just six weeks, the complete population tabulation was finished in about six months, and detailed processing in two years (the complete publication of the census took several more years). Hollerith's system saved an estimated $5 million in labor costs compared with manual processing (the 1890 census itself cost $11.5 million — almost double the 1880 census, but with an incomparably larger volume of processed data). Hollerith's company, the Tabulating Machine Company (founded in 1896), merged in 1911 with other firms into the Computing-Tabulating-Recording Company (CTR), renamed IBM in 1924.

Hollerith's cards from 1890 had a format of 12 × 24 (12 rows, 24 columns) with round holes. The format gradually grew — around 1910 to 45 columns. In 1928 IBM introduced the revolutionary 80-column format with rectangular holes — the card measured exactly 7⅜ × 3¼ inches (187.325 × 82.55 mm), 80 columns and originally 10 numeric rows (0–9); in 1930 two "zone" rows (11 and 12) were added for encoding letters and special characters, creating the definitive 80 × 12 format. One column = one character. A group of consecutive columns reserved for a specific purpose (name, address, employee number) was called a field. A card with fixed identification data for a group of cards was a master card, the others were detail cards.

This format dominated the industry for more than four decades. As late as the mid-1950s, punched card sales accounted for 20% of IBM's revenue and 30% of its profit. The legacy persists to this day — the 80-character line in terminals (IBM 3270) and programming conventions (FORTRAN) is a direct remnant of the punched card's dimensions.

Before the era of electronic computers, there existed an entire ecosystem of electromechanical machines for processing data on punched cards, collectively called unit record equipment (or tabulating machines). The fundamental principle: one card = one record (unit record). The machines included:

Processing was purely batch and sequential: the operator prepared a deck of cards, inserted it into the machine, which processed them one by one in a prescribed order. For more complex calculations, the output deck of one machine became the input of the next — effectively a processing pipeline, where each phase corresponded (in a modern analogy) to an SQL clause: sorting = ORDER BY, selection = WHERE, summarization = GROUP BY.

Record numbers (sequence numbers) — typically in the last columns of the card (73–80) — served to identify the order in the deck. If the operator dropped the deck (which happened), the sorter sorted it back according to the sequence numbers. This convention survived in FORTRAN, where columns 73–80 were reserved for sequence numbers and were not part of the program code.

All data storage and processing from the 1950s to the present takes place within the von Neumann architecture — the model described in the document "First Draft of a Report on the EDVAC" (June 1945), written by the mathematician John von Neumann on the basis of discussions with J. Presper Eckert and John Mauchly at the University of Pennsylvania. The key concept: the stored program — instructions and data share the same memory and are addressable by the same mechanism. The computer consists of a processor (arithmetic-logic unit + control unit), memory, and input and output devices.

This model had a direct impact on the development of data media, because it introduced a fundamental distinction between:

The von Neumann bottleneck: because instructions and data share a single bus between the processor and memory, the processor is forced to wait for data from memory — and the gap between processor speed and memory speed has steadily deepened since the 1950s. The answer became the memory hierarchy — from the fastest and most expensive (CPU registers) through ever slower and cheaper levels (cache → RAM → disk → tape → archive). Every data storage technology this document discusses is one layer of this hierarchy.

The magnetic drum, invented by the Austrian Gustav Tauschek in 1932, was a rotating metal cylinder coated with ferromagnetic material with a row of fixed read/write heads — one per track. Tauschek's prototype stored 500,000 bits (about 62.5 KB). In the 1950s magnetic drums became the main operating memory of a series of computers — these machines were characteristically called "drum computers" or "drum machines."

The best-known example is the IBM 650 (1954) — the world's first mass-produced computer (almost 2,000 units produced). Its operating memory was a magnetic drum with a capacity of 2,000 ten-digit words (about 17.5 KB), later doubled to 4,000 words. When coding, the programmer had to take into account the drum's rotational latency — the optimal placement of the next instruction at the drum position that would be under the read head precisely at the moment the processor needed it. The SOAP assembler (Symbolic Optimal Assembly Program) performed this optimization automatically. The IBM 650 ran at a frequency of 125 kHz and the average instruction time was 27.6 ms (about 40 instructions per second).

Other drum computers included the ERA Atlas / UNIVAC 1101 (1950, developed for cryptanalysis for the US Navy — the ERA drum stored 16,384 words), the Manchester Mark 1, the British DEUCE (a commercial version of Turing's ACE design by English Electric), and the Japanese ETL Mark IV (1957).

Magnetic drums as main memory were displaced in the 1960s by ferrite cores (magnetic core memory, developed by Jay Forrester at MIT as part of the Whirlwind project in 1949–1953) — the first reliable high-speed random-access memory with no moving parts. Drums then moved into the role of fast auxiliary memory (swapping, paging) — to this day BSD Unix has a /dev/drum device as a reference to that era. Ferrite cores dominated until the 1970s, when they were replaced by semiconductor DRAM memory (Intel 1103, 1970 — the first commercial DRAM chip, 1 Kbit).

The development of data media directly reflects the layers of the von Neumann hierarchy:

1940 — main memory: delay lines (mercury); external memory: none; archive/input: punched cards, paper tape.

1950 — main: magnetic drums; external: magnetic tapes; archive: punched cards.

1960 — main: ferrite cores; external: magnetic disks + tapes; archive: punched cards (declining).

1970 — main: DRAM (semiconductors); external: hard disks; archive: magnetic tapes.

1980–90 — main: DRAM + cache (SRAM); external: HDD + SSD (emerging); archive: tapes, optical media.

2000+ — main: DRAM + L1/L2/L3 cache; external: SSD + HDD; archive: tapes (LTO), cloud.

2020 — main: DRAM + persistent memory; external: NVMe SSD + object storage; archive: LTO-9/10, cold storage.

The key pattern: a technology that serves as main memory in one era moves in the following era to the position of external memory or archive. Magnetic drums (1950s main → 1960s auxiliary), ferrite cores (1960s main → 1970s replaced), magnetic disks (1960s online storage → still secondary today), magnetic tapes (1950s primary storage → archive today).

Databases in each era optimize data access in the context of the current layer of the hierarchy. ISAM and B-trees were designed for the specifics of rotating disks (seek time + rotational latency), the LSM-tree (RocksDB) for SSDs (random writes → sequential), in-memory databases (VoltDB, Redis) eliminate the disk layer entirely.

On March 31, 1951, the US Census Bureau accepted the first UNIVAC I (Universal Automatic Computer I) — the first American computer designed for commercial data processing, developed by J. Presper Eckert and John Mauchly (the creators of ENIAC).

The key innovation was the UNISERVO — the first commercial tape drive. The medium was a 0.5-inch tape of nickel-plated phosphor bronze, up to 1,500 feet (457 m) long, with a recording density of 128 characters per inch across 8 tracks (6 data + 1 parity + 1 timing). A linear speed of 100 inches/s gave a nominal transfer rate of 12,800 characters/s (in reality about 7,200 char/s after accounting for inter-block gaps). About 1.44 million decimal digits fit on a single tape. The UNIVAC I could have up to 10 tape units connected and supported reading and writing in both directions — a key feature for efficient sorting and merging.

The UNIVAC I became famous on November 4, 1952, when for CBS it correctly predicted Eisenhower's victory in the presidential election from a sample of only 5.5% of the votes — millions of Americans saw an electronic computer in action for the first time (in reality only a terminal in New York; the machine itself was in Philadelphia).

In 1952 IBM followed with the 726 tape unit for the IBM 701 computer — this already used a plastic tape with a ferric-oxide coating (developed by 3M to IBM's specifications) and a revolutionary vacuum column (patent of James Weidenhammer and Walter Buslik) to cushion the rapid starts and stops of the tape without breaking it. The IBM 726 achieved a density of 100 characters/inch and a speed of 75 inches/s. The plastic tape and the vacuum column became the industry standard.

Data on tape was organized into blocks (physical records) separated by an inter-block gap (IBG) — typically 0.6–0.75 inches of blank tape allowing the mechanism to stop and start. One block contained one or more logical records. The number of logical records in a block was called the blocking factor — a higher factor meant more efficient use of the tape (fewer gaps) but required a larger buffer.

Each logical record had a fixed length (fixed-length record) defined in the program — for example 80 bytes (corresponding to one punched card), 132 bytes (the width of a line printer), or another value depending on the application.

Records within a file had no names; they were identified by a sequence number (relative record number — the position in the sequence) or by the value of a key field — for example employee number, account number, or part number. There was no schema separate from the program — the structure of the record (which bytes represented which field) was "hard-wired" into the COBOL DATA DIVISION, a PL/I structure declaration, or the assembly code of each application. Changing the record format (adding a field, changing a length) required converting the entire tape and rewriting all programs working with the data.

Tape files had a header label at the beginning containing the file identification, creation date, record count, and other metadata, and a trailer label with checksums. IBM standardized the label format (Standard Labels) for data exchange between installations.

Magnetic tape is a purely sequential medium — to access a record in the middle, one must rewind on average a third of the tape's length, which takes tens of seconds. This fundamental limitation defined the data processing paradigm for two whole decades:

The master file contained the current state of the data — for example a list of all employees with their wages, hours worked, and deductions. It was sorted by the key field (employee number). The transaction file contained the changes — new employees, terminations, address updates, hours worked for the period. It was likewise sorted by the same key field.

Processing was carried out by sequential merge/update: the program read both files synchronously from two tape units, compared keys, and created a new master file on a third (output) tape:

The old master file was kept as a backup — giving rise to the so-called generation principle (generation data groups, GDG): "grandfather," "father," "son." If processing failed, the computation was repeated from the previous generation of the master file. The entire master file had to be rewritten with every processing run, even if only a fraction of the records changed. Direct access to a single record did not exist — to find out the status of one employee, it was necessary to sequentially go through the entire tape. This was acceptable for batch processing (payroll once a week, billing once a month), but completely unsuitable for interactive querying.

A typical configuration required at least 4 tape units: input master, input transactions, output new master, and output report/error log. Larger installations had 8–16 tape units for parallel processing and multi-level sorting (a merge sort on tapes required at least 3 tapes for two-way merging, optimally 6+ for multi-way merging).

The breakthrough came in September 1956, when IBM introduced the RAMAC 305 (Random Access Method of Accounting and Control) — the first commercial hard disk. It contained 50 rotating platters 24 inches in diameter with a capacity of 5 million characters (about 5 MB). The access time was 600 ms — slow by today's standards, but revolutionarily fast compared with the minutes of rewinding tape. For the first time it was possible to access a single record directly without sequentially going through the entire file.

This opened the way to new access methods:

The existence of disks made possible the transition from purely batch processing to online processing (OLTP) — an end user at a terminal could enter a query and get an answer in seconds, rather than waiting for the nightly batch. This need for direct access to data and for sharing it between applications created demand for database management systems (DBMS).

The term "data-base" first appeared in a technical sense in 1962 in a report by the System Development Corporation in California. In the 1960s each application had its own files in its own format — data redundancy (the same data stored multiple times in different files), data inconsistency (after updating one copy, the others remained out of date), the absence of data sharing between applications, no query language, no guarantee of integrity. Changing a file's structure required rewriting all the programs that worked with it. This situation gave rise to the need for centralized data management — a database management system (DBMS).

The hierarchical database model arose from a specific practical need. In May 1961 President Kennedy announced the Apollo program; the Saturn V rocket was estimated to have over 3 million components, the tracking of which was beyond the capabilities of file systems. In 1963 IBM and North American Aviation (later Rockwell) began collaborating on parts tracking (a formal partnership was concluded in 1965) and, with the participation of Caterpillar (as an early commercial user), developed the ICS/DL/I system — the team consisted of 12 developers from IBM, 10 from Rockwell, and 3 from Caterpillar. On August 14, 1968, the system displayed its first "READY" message at NASA Downey in California. In 1969 it was renamed IMS/360 and brought to market. The chief architect was Vern Watts, informally called the "father of IMS."

IMS organizes data into a tree structure — each database record consists of a root segment and its subordinate segments, the hierarchy can have up to 15 levels and 255 segment types. Programmers access the data through the DL/I language (Data Language/Interface) using calls such as GU (Get Unique), GN (Get Next), ISRT (Insert), or DLET (Delete). The application's view of the database is defined by the PCB (Program Communication Block) with a specification of sensitive segments and permitted operations. The physical structure is described by the DBD (Database Description).

IMS's strengths — extreme performance for predefined access patterns, built-in recovery and integrity — ensured its survival to this day. In 2003 IMS had its most successful business year (35 years after its release), and as of 2022 it was still used by 95% of Fortune 1000 companies. The weakness was the rigid tree: each child could have only one parent, there were no ad hoc queries, and changing the data structure required reprogramming the applications.

In parallel with IMS, the network model was emerging. Charles W. Bachman, in 1961–1963 at General Electric, developed the Integrated Data Store (IDS) — considered the first general-purpose database management system (DBMS). Unlike the hierarchical model, IDS organized data into a graph structure with pointer chains — a record could be a member of multiple sets, thereby naturally modeling many-to-many relationships. Bachman also introduced Bachman diagrams for the visual representation of data structures.

For his contribution Bachman received the Turing Award in 1973 — he was the first laureate without a Ph.D. and the first with an entire career in industry. His lecture "The Programmer as Navigator" defined the concept of the navigational database: the programmer moves (navigates) through the database via pointers.

The standardization effort was taken up by CODASYL (Conference on Data Systems Languages), originally founded in 1959 to develop COBOL. In 1965 the List Processing Task Force was created, renamed in 1967 the Data Base Task Group (DBTG). The key DBTG Report of April 1971 defined the architecture of the network database and introduced terms used to this day: DDL (Data Definition Language), DML (Data Manipulation Language), schema and subschema, data independence. The network model was implemented by products such as IDMS (Cullinet/CA, for IBM mainframes — in 2000 still running on more than 1,000 mainframes), IDS/2 (Honeywell), DMS-1100 (Univac), and HP IMAGE.

The disadvantage of the network model was its complexity — the programmer had to explicitly traverse pointer chains record by record, without a declarative query language. The absence of a mathematical foundation made automatic optimization of access paths impossible. It was precisely these shortcomings that the relational model came to address.

Edgar Frank "Ted" Codd (1923–2003), a British computer scientist working at the IBM San Jose Research Laboratory, published in June 1970 the paper "A Relational Model of Data for Large Shared Data Banks" in Communications of the ACM (vol. 13, no. 6, pp. 377–387). The paper presented a radically new approach: data is organized as mathematical relations (sets of tuples), represented as tables with rows and columns. The key contributions included data independence (the independence of applications from the physical storage), relational algebra for formal manipulation of data, and the elimination of the navigational approach.

In 1971 Codd supplemented the theory with normal forms (1NF, 2NF, 3NF), and in 1974 he and Raymond Boyce defined BCNF (Boyce-Codd Normal Form). Ronald Fagin later added 4NF (1977) and 5NF (1979). Codd received the Turing Award in 1981 and in 1985 published the famous 12 rules (in reality 13, numbered 0–12) defining what is truly a relational database — no commercial system fully satisfies them to this day.

A pivotal moment was the Bachman–Codd debate at the SIGFIDET conference in Ann Arbor in 1974 — before it, the network model was the mainstream and the relational approach was the "challenger"; after it, the trend began to turn.

System R (1973–1979) arose at the IBM San Jose Research Lab — the team of about 15 people included Don Chamberlin and Raymond Boyce, who designed the SEQUEL language (later renamed SQL because of the Hawker Siddeley trademark). Patricia Selinger developed a groundbreaking cost-based query optimizer — a technique used by all modern databases. Jim Gray formalized the concepts of serializability, two-phase locking, and transaction processing. Paradoxically, IBM isolated the development team from Codd — they did not use his Alpha language but created their own SQL.

INGRES (1973–1979) at UC Berkeley was led by Michael Stonebraker and Eugene Wong — it never had more than 5–6 programmers, ran on a PDP-11 under UNIX, and used the QUEL language. QUEL was considered a mathematically cleaner alternative to SQL, but SQL won out thanks to its easier readability. Both projects received the ACM Software Systems Award in 1988 and demonstrated the practical viability of the relational model.

SQL-86 (1986, 120 pages) — SELECT, INSERT, UPDATE, DELETE, GROUP BY, subqueries. SQL-89 (1989) — PRIMARY KEY, FOREIGN KEY, CHECK, bindings for C and Ada. SQL-92 (1992, 579 pages) — explicit JOIN syntax, CASE, DATE/TIME, ALTER, DROP. SQL:1999 (1999, modular) — CTE, recursive queries, triggers, UDT, BOOLEAN, ROLLUP/CUBE. SQL:2003 (2003) — window functions, XML, MERGE, SEQUENCE, MULTISET. SQL:2011 (2011) — temporal tables (system-versioned). SQL:2016 (2016) — JSON support, MATCH_RECOGNIZE, polymorphic table functions. SQL:2023 (2023) — SQL/PGQ (property graph queries), extended JSON, GREATEST/LEAST.

SQL:2023, adopted in June 2023, introduces a new Part 16 — SQL/PGQ — enabling the querying of relational data as graphs. With this, the SQL standard formally moves closer to graph databases and reduces the need for specialized graph systems.

Larry Ellison in 1977 founded Software Development Laboratories (later Oracle) after reading the publicly available papers on System R. Oracle V2 was released in 1979 as the first commercially available SQL RDBMS. IBM introduced SQL/DS in 1981 and DB2 in 1983.

PostgreSQL evolved from the academic INGRES through Stonebraker's POSTGRES project (1986) and Postgres95 (1994, when Andrew Yu and Jolly Chen added SQL) after being renamed in 1996 — today it is one of the most popular databases in the world. MySQL was created by Michael "Monty" Widenius and David Axmark, with the first release on May 23, 1995; in 2008 Sun Microsystems bought it for $1 billion, in 2010 it passed to Oracle, and Widenius responded with the MariaDB fork.

Microsoft SQL Server arose from a partnership with Sybase (1988); the code from Sybase was completely rewritten only in SQL Server 2005. SQLite, created by D. Richard Hipp in 2000 for the US Navy (a damage-control system on the destroyer USS Oscar Austin), is today the most widespread database in the world — running on billions of devices.

The acronym ACID was coined by Theo Härder and Andreas Reuter in their 1983 paper "Principles of Transaction-Oriented Database Recovery." They built on the work of Jim Gray, who in 1981 named atomicity, consistency, and durability (but not isolation). Gray — a Turing Award holder (1998) — formalized serializability, two-phase locking, Write-Ahead Logging (WAL), and two-phase commit. These concepts remain the foundation of transaction processing to this day. Of note is that IBM's IMS supported ACID transactions as early as 1973 — ten years before the acronym was coined.

Object-oriented databases (OODBMS) attempted to solve the so-called impedance mismatch — the fundamental conceptual gap between object-oriented programming (objects, methods, inheritance, reference graphs) and the relational model (tables, rows, joins, SQL). The ODMG standard (Object Data Management Group), founded in the summer of 1991 at the initiative of Rick Cattell of Sun Microsystems with five key OODBMS vendors, defined an object model, the OQL query language, and bindings for C++, Smalltalk, and Java. Five revisions were published (ODMG 1.0 in 1993 to ODMG 3.0 in 2000), after which the group dissolved.

Key products included ObjectStore (Object Design, Inc., strong in CAD/telco), Versant (telecom, finance, defense), GemStone/S (Smalltalk, insurance, transportation), db4o (open-source Java/C#), and InterSystems Caché — the commercially most successful OODBMS, dominant in healthcare. OODBMSs gained ground in CAD/CAM, telecommunications, scientific computing, and healthcare, but did not achieve mass adoption.

The reasons for the failure were manifold: binding to a specific compiler (different C++ compilers stored objects in incompatible formats), the absence of a universal query language comparable to SQL, weak support for ad hoc queries and reporting, fragmentation of the market among a dozen small vendors without the backing of the big players (Oracle and IBM stayed with the relational model), and the enormous existing investments of organizations in relational databases. Approximately 70% of OODBMS revenue came from professional services, which indicated a strong vendor dependence.

A more successful solution to the impedance mismatch became the object-relational model, which extended the existing SQL and relational foundations with object elements. The key example is PostgreSQL, directly continuing Stonebraker's POSTGRES project (1986), whose goal was to add user-defined data types, inheritance, and extensibility to the relational model.

The SQL:1999 standard formally enshrined user-defined types (UDT), type inheritance, reference types (REF), collections (arrays, multisets), and stored procedures. Oracle, IBM DB2, and SQL Server all implemented object-relational extensions with varying degrees of conformance to the standard. The fundamental advantage was backward compatibility — organizations could add object elements gradually without abandoning the relational foundation and their investment in SQL.

The explosion of Web 2.0 in the mid-2000s brought a volume, variety, and velocity of data exceeding the capabilities of traditional RDBMSs. Three technical papers from Google and Amazon changed the paradigm: Google MapReduce (2004, Jeffrey Dean and Sanjay Ghemawat) — a model for parallel processing; Google Bigtable (2006) — a columnar distributed system inspiring HBase and Cassandra; and Amazon Dynamo (2007) — a master-less key-value store with consistent hashing and eventual consistency, inspiring Riak and DynamoDB.

The term "NoSQL" in its modern sense originated on June 11, 2009, when Johan Oskarsson of Last.fm needed a short hashtag for a meetup in San Francisco devoted to "open-source distributed, non-relational databases" — on the IRC channel #cassandra someone suggested "NoSQL." The label was not meant to last, but it spread virally. Later the interpretation "Not Only SQL" became established.

Eric Brewer of UC Berkeley presented his conjecture in a keynote at the PODC conference in July 2000 in Portland: a distributed data system can simultaneously guarantee at most two of three properties — consistency (every read returns the most recent write), availability (every request to a functioning node receives a response), and tolerance to network partitioning (the system functions even when communication between nodes fails). In 2002 Seth Gilbert and Nancy Lynch of MIT published a formal proof. Because network partitions are inevitable in distributed systems, the real choice is between CP (consistency + tolerance) and AP (availability + tolerance).

Key-Value stores offer the simplest model — a key mapped to a value. Redis (REmote DIctionary Server), created by Salvatore Sanfilippo and released open-source on April 10, 2009, supports rich in-memory data structures (strings, hashes, lists, sorted sets). Amazon DynamoDB (a cloud service since 2012) offers a fully managed key-value/document store. These systems excel at caching, session management, and real-time leaderboards.

Document stores store semi-structured data as documents (typically JSON/BSON). MongoDB (developed from 2007 at the company 10gen, first public release 2009) became the most popular — it stores documents in BSON, supports a dynamic schema, sharding, and from version 4.0 (2018) also multi-document ACID transactions. Apache CouchDB, created by Damien Katz in April 2005 (originally in C++, rewritten in Erlang in 2006), offers a RESTful API and multi-master replication for offline-first scenarios.

Column-family (wide columns) combine the BigTable and Dynamo approaches. Apache Cassandra, originally developed at Facebook in 2008 for inbox search (Avinash Lakshman and Prashant Malik), combines the Bigtable data model with the master-less Dynamo architecture. It is an AP system with tunable consistency. HBase is an open-source implementation of BigTable on top of Hadoop (a CP system). ScyllaDB is a rewrite of Cassandra in C++, eliminating the pauses of the JVM garbage collector.

Graph databases model data as nodes and edges. Neo4j, conceptualized around 2000 by Emil Eifrem in Sweden (version 1.0 in February 2010), uses the property graph model and the Cypher query language. It became famous for the analysis of the Panama Papers in 2016. In June 2021 it reached a valuation of over $2 billion.

NoSQL systems often work with the BASE model — Basically Available (the system responds even during partial outages), Soft state (the state can change over time even without external input), Eventually consistent (after all updates have propagated, the replicas converge). The acronym BASE was deliberately chosen as the opposite of ACID (base vs acid in chemistry). Dan Pritchett of eBay popularized it in an article for ACM Queue in 2008. ACID is suitable for financial systems and healthcare, where data integrity is critical; BASE for social networks and e-commerce at large scale, where availability is the priority.

The term NewSQL was coined by Matthew Aslett of 451 Research on April 6, 2011 as shorthand for new scalable SQL databases that combine the ACID guarantees of traditional RDBMSs with the horizontal scalability of NoSQL.

Google Spanner, described in a paper at OSDI 2012, represents a breakthrough system — the first globally distributed database with strong consistency. The key innovation is the TrueTime API, which uses GPS receivers and atomic clocks in every datacenter to provide globally synchronized timestamps with bounded uncertainty (typically under 10 ms). Spanner achieves external consistency — the strongest consistency property for transactional systems — through a commit-wait phase, in which it waits until TrueTime confirms that the assigned timestamp is in the past. It powers Gmail and YouTube and is available as Google Cloud Spanner.

CockroachDB, founded in 2015 by ex-Googlers Spencer Kimball, Peter Mattis, and Ben Darnell (all of whom worked with Bigtable/Spanner), offers a distributed SQL database with the PostgreSQL wire protocol — achieving consistency similar to Spanner, but without specialized hardware (it uses hybrid logical clocks).

TiDB from PingCAP (2015) is MySQL-compatible with a two-layer architecture (an SQL layer + TiKV storage with Raft consensus) and HTAP support thanks to the columnar TiFlash engine. VoltDB, the commercialization of the H-Store research project under the leadership of Michael Stonebraker (Turing Award 2014), is an in-memory ACID database achieving 3 million transactions per second on 384 cores thanks to the elimination of locking, WAL, and buffer pool management.

By 2025 the term NewSQL had largely transformed into "Distributed SQL" — a 451 Research survey showed that 58% of IT decision-makers expected to deploy a globally distributed database within one year.

General-purpose RDBMSs can model time series, but they are not optimized for sustained high-frequency data ingestion or large-scale analytical scans. InfluxDB (2013, InfluxData) uses the TSM engine (Time-Structured Merge Tree) and in version 3.0 moved to a columnar architecture with Apache Arrow and Parquet. TimescaleDB (2017) is a PostgreSQL extension introducing the concept of hypertables — automatic partitioning of time-series data into chunks. In June 2025 the parent company Timescale renamed itself TigerData. QuestDB, written in low-latency Java, C++, and Rust, in benchmarks achieves 12–36× higher performance than InfluxDB 3 Core for data ingestion and 43–418× faster analytical queries. Typical deployments include IoT, infrastructure monitoring, and financial market data.

Vector databases store high-dimensional vector embeddings (numerical representations of text, images, audio) and perform approximate nearest neighbor (ANN) search. Key indexing structures include HNSW (Hierarchical Navigable Small World — a graph algorithm with logarithmic complexity) and IVF (Inverted File Index — clustering of vectors). The vector database market grew from $1.73 billion in 2024 to a projected $10.6 billion by 2032.

For RAG (Retrieval-Augmented Generation) vector databases are essential: content is broken down into segments, converted into vector embeddings, stored in the database, and at query time relevant fragments are found by semantic search to enrich the LLM's response.

Key products:

In 2025 vector search became a commodity feature — traditional databases (PostgreSQL, MongoDB, Oracle, Elasticsearch) integrate it natively. A VentureBeat article from November 2025 noted that dedicated vector databases are losing differentiation, because the vector index is becoming just another type of index alongside B-tree and GIN/GiST.

SQLite remains the most widespread database in the world with billions of instances — running in every smartphone (Android and iOS), browser, and IoT device. Its "entire database in a single file" design is ideal for embedded and edge scenarios.

DuckDB, created in 2018 by Mark Raasveldt and Hannes Mühleisen at CWI in Amsterdam (Centrum Wiskunde & Informatica — the institution where MonetDB originated and where Guido van Rossum created Python), represents a revolution in analytics. It is the "SQLite for analytics" — an in-process OLAP database with columnar storage and vectorized query processing. DuckDB is 10–50× faster than SQLite for analytical queries, directly reads Parquet, CSV, JSON, and Arrow files without importing, and runs anywhere — from a Jupyter notebook to WebAssembly in a browser. In 2025 it reached 6 million PyPI downloads per month. A DuckDB 1.4 benchmark showed a 100× speedup over Apache Spark for typical BI queries on medium-sized datasets.

RocksDB (Meta/Facebook) is an embedded key-value store based on the LSM-tree (Log-Structured Merge Tree), optimized for SSDs with high write throughput. It is not a database for end users, but a storage engine used inside other databases — TiKV (under TiDB), CockroachDB, YugabyteDB, and many others.

ArangoDB combines document + graph + key-value in a single engine with the unified AQL query language. SurrealDB (Rust, $44 million in funding) attempts the most ambitious integration — relational, document, graph, time-series, key-value, and vector models in one. At the opposite end of the spectrum stands the cautionary tale of FaunaDB — a multi-model serverless database that ceased operations on May 30, 2025. At its peak Fauna claimed it was used by over 80,000 developer teams; when the service shut down, hundreds of paying customers and thousands of teams had to migrate urgently. This incident underscored the risk of vendor lock-in with proprietary cloud databases.

The serverless model shifts infrastructure management entirely to the provider — the user pays only for consumption, the database scales automatically, including scaling to zero.

Results of the Stack Overflow Developer Survey 2025: 55.6% of developers use PostgreSQL (the highest share of all databases, the third year as #1 in the "most used," "most loved," and "most wanted" categories), MySQL 40.5%. The year 2025 brought an unprecedented wave of acquisitions in the PostgreSQL ecosystem:

PostgreSQL 18 (released September 25, 2025) brought fundamental technical improvements: an asynchronous I/O subsystem (AIO) dramatically speeding up sequential scans and checkpointing, skip scans for more efficient use of multi-column indexes, query optimizer improvements, and extended JSON support.

PostgreSQL's strength lies in its ecosystem of extensions — PostGIS (geospatial data), pgvector (vector embeddings), Citus (horizontal scaling), TimescaleDB (time series), pg_duckdb (OLAP analytics), ParadeDB (full-text search with BM25) — thanks to which a single database covers OLTP, OLAP, vectors, graphs, geospatial, and time-series data. The question of 2025 is no longer "relational or non-relational?", but "which extension do I need?".

DuckDB recorded a jump in the DB-Engines ranking from position #81 to #51 with 50.7% year-on-year growth — the fastest-growing database in the ranking. Key milestones:

DuckDB established itself in 2025 as a key player in the analytics ecosystem — it inspired a new wave of "in-process" analytics tools and its adoption continues to accelerate.

The lakehouse architecture, where data is stored in open formats on object storage (S3/GCS/ADLS) with ACID transactional guarantees, became mainstream in 2025. "S3 is the new disk" — this statement sums up the shift from proprietary storage engines to commodity object storage.

The three main open table formats:

Convergence with streaming systems (Kafka + Flink → Iceberg) makes it possible to build a real-time lakehouse — data flows from operational systems through a streaming platform directly into Iceberg tables, where it is immediately available for analytical queries.

The year 2025 brought a fundamental shift in how databases interact with AI systems:

Oracle AI Database 26ai (October 2025) introduced the concept of the "Select AI Agent" — AI agents as "first-class citizens" of the database, which can autonomously generate and run SQL queries, optimize schemas, and answer questions in natural language directly over the database.

Google Cloud introduced HTAP+V — the convergence of transactional, analytical, and vector processing in a single database. TiDB (PingCAP) began positioning itself as an "AI-native database for millions of autonomous agents" — each agent has its own context window enriched with structured and unstructured data from TiDB.

A key innovation of 2025 became the Model Context Protocol (MCP) — an open standard for connecting AI agents to data sources, described as "the new ODBC for the AI era." MCP defines a standardized interface through which AI models access databases, APIs, and other data sources without the need for a specific integration for each system. By the end of 2025 MCP was supported by Oracle, Databricks, Snowflake, and dozens of other platforms.

A VentureBeat article from January 2026 noted that "RAG is dead in its original form" — the simple pipeline architecture (chunk → embed → retrieve → generate) is being replaced by agentic memory systems, where AI agents autonomously decide when and how to access data sources, combine vector search with SQL queries, and maintain persistent memory across sessions.

Herman Hollerith (1860–1929) — the punched card, the tabulator, founder of the company → IBM.

Edgar F. Codd (1923–2003) — the relational model (1970), normal forms, the 12 rules. Turing Award 1981.

Charles W. Bachman (1924–2017) — IDS (the first DBMS), the network model, CODASYL DBTG. Turing Award 1973.

Michael Stonebraker (1943–) — INGRES, POSTGRES/PostgreSQL, VoltDB. Turing Award 2014.

Jim Gray (1944–2007*) — System R, transaction theory (serializability, WAL, 2PC). Turing Award 1998.

Patricia Selinger — the cost-based query optimizer (System R). ACM Fellow.

Eric Brewer (1966–) — the CAP theorem (2000), Google infrastructure. ACM Prize 2009.

Don Chamberlin (1944–) — co-creator of SQL (SEQUEL, 1974). ACM Fellow.

*Jim Gray disappeared while sailing near San Francisco in January 2007, declared dead in 2012.

The history of database models reveals a recurring pattern: each generation solves the shortcomings of the previous one, but introduces its own trade-offs:

1. Convergence of models. The boundaries between SQL, NoSQL, graph, and vector databases are dissolving. PostgreSQL with extensions covers almost all use cases. SQL:2023 with SQL/PGQ brings graph querying into the relational world.

2. Data as infrastructure for AI. Vector search has become a commodity. MCP is replacing ODBC as the standard interface. Databases are transforming from "systems of record" into "systems of reasoning."

3. Decentralization of computation. Edge databases (Turso, D1), in-process analytics (DuckDB), lakehouse formats (Iceberg, DuckLake) move processing closer to the data and to users. Streaming and batch processing are converging.

The database of 2026 is no longer a single product, but an ecosystem of interconnected layers:

The developer at the start of 2026 does not choose between a relational and a non-relational database — they choose which extension to add to PostgreSQL and which layers of the ecosystem to connect.

Methodological note

The conception, structure, and editorial line of the article are the work of the author, who prepared the content sketch, established the key theses, and directed the entire creative process. Generative AI (Claude, Anthropic) was used as a technical tool for research, fact-checking, and fleshing out the author's outline.

The author continuously edited the outputs, verified the key findings, and approved the final wording. No part of the text was published without human review. All factual data were verified against the publicly available sources cited in the text.

The procedure complies with the requirements of Art. 50 of EU Regulation 2024/1689 (the AI Act) on the transparency of AI-generated content. #poweredByAI