The Apache Software Foundation recently announced a new open-source project named Apache Arrow, that provides a high-performance data layer for columnar in-memory analytics.
The Apache Arrow enables execution engines to take advantage of the latest SIMD (Single input multiple data) operations included in modern processors, for native vectorized optimization of analytical data processing. Columnar layout of data also allows for a better use of CPU caches by placing all data relevant to a column operation in as compact of a format as possible.
The Apache Arrow acts as a new high-performance interface between various systems. It is also focused on supporting a wide variety of industry-standard programming languages. Java, C, C++, Python are underway and more languages are expected soon.
Learn more about Apache Arrow: https://arrow.apache.org/