Xorbits: scalable Python data science, familiar & fast.

Scalable Python data science

Familiar & Fast

Xorbits is a scalable Python data science framework that aims to scale the whole Python data science world, including numpy, pandas, scikit-learn and many other libraries.

View on GitHub Join Community

Core Features

Fastest distributed framework for data science

Xorbits can leverage multi cores or GPUs to accelerate computation on a single machine, or scale out up to thousands of machines to support processing terabytes of data.

View benchmarks

View documentation

Familiar APIs

No modification of your code except for import replacements

Massive Datasets

Xorbits can easily process terabytes of data.

Lightening Fast

Xorbits is the fastest framework among the most popular distributed data science frameworks

Batteries Included

Xorbits can run everywhere, laptop, cluster, Kubernetes, cloud and so forth.

Scale Pandas, numpy, and many other libraries

Just replacing import with Xorbits, e.g. replace import pandas with import xorbits.pandas. Your code may be shockingly fast and able to process massive datasets.

Fully compatible APIs including pandas, numpy and many other libraries
Ability to process massive datasets
Deploy everywhere, locally, distributed, or on the cloud

Easily process massive datasets

Xorbits can process terabytes of data. Xorbits supports for many data formats, including Parquet, CSV, Databases, HDF5, Zarr, and so forth.

Not only Pandas,
but also numpy, scikit-learn and many other libraries

One of the missions of Xorbits is to scale the entire Python data science world, not only pandas is scalable inside Xorbits, many other libraries like numpy, scikit-learn are ready to scale as well.

Easily deploy everywhere

Don't worry, you don't need to change your code, they can run everywhere, locally, on a cluster, or on cloud.