Scalable Python data science
Familiar & Fast
Xorbits is a scalable Python data science framework that aims to scale the whole Python data science world, including numpy, pandas, scikit-learn and many other libraries.
Xorbits can leverage multi cores or GPUs to accelerate computation on a single machine, or scale out up to thousands of machines to support processing terabytes of data.
No modification of your code except for import replacements
Xorbits can easily process terabytes of data.
Xorbits is the fastest framework among the most popular distributed data science frameworks
Xorbits can run everywhere, laptop, cluster, Kubernetes, cloud and so forth.
Just replacing import with Xorbits, e.g. replace import pandas with import xorbits.pandas. Your code may be shockingly fast and able to process massive datasets.
Xorbits can process terabytes of data. Xorbits supports for many data formats, including Parquet, CSV, Databases, HDF5, Zarr, and so forth.
One of the missions of Xorbits is to scale the entire Python data science world, not only pandas is scalable inside Xorbits, many other libraries like numpy, scikit-learn are ready to scale as well.
Don't worry, you don't need to change your code, they can run everywhere, locally, on a cluster, or on cloud.
Find out what we've been up to at Xorbits -- news,tutorials,workshops,events and more