Welcome to the exciting world of the Intel® Scalable Dataframe Compiler (Intel® SDC). This powerful extension of Numba enables the compilation of Pandas operations, automatically vectorizing and parallelizing your code. Let’s dive into how you can install, compile, and troubleshoot Intel® SDC seamlessly!

Installation of Intel® SDC

Getting Intel® SDC up and running involves a few straightforward steps. You can either install it via conda or wheel package formats. Here’s how:

1. Using Conda

• First, create a new conda environment:

conda create -n sdc-env python=3.6 or 3.7 -c anaconda -c conda-forge

• Activate the environment:

conda activate sdc-env

• Install Intel® SDC:

conda install sdc -c intellabelbeta -c intel -c defaults -c conda-forge --override-channels

2. Using Wheel Package

• First, create a new environment with pip:

conda create -n sdc-env python=3.6 or 3.7 pip -c anaconda -c conda-forge

• Activate the environment:

conda activate sdc-env

• Then, install the SDC:

pip install --index-url https:pypi.anaconda.org/intellabelbeta/simple --extra-index-url https:pypi.anaconda.org/intel/simple --extra-index-url https:pypi.org/simple sdc

Building Intel® SDC from Source

Building Intel® SDC from source can be performed on both Linux and Windows. Here, we will use a simple analogy: Imagine you want to build your own car instead of buying a ready-made one. You will require a robust set of tools and a solid workbench. Similarly, to build Intel® SDC, you must have the right environment and dependencies set up on your operating system.

Building on Linux

• Set up your Anaconda distribution. If you don’t have conda, you can use Miniconda:

wget https:repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh

• After making it executable, install it:

chmod +x miniconda.sh
    ./miniconda.sh -b
    export PATH=$HOME/miniconda3/bin:$PATH

• Next, use conda-build or setuptools to build the compiler following the instructions in the README file.

Building on Windows

• Ensure you have Build Tools for Visual Studio 2019. Install it from here.
• Install Miniconda for Windows from this link.
• Start the Anaconda prompt and follow similar conda-build or setuptools steps as mentioned for Linux.

Troubleshooting

As with any development journey, you might run into a few bumps along the way. Common issues and their resolutions include:

• If you encounter an error regarding the ‘cl’ compiler not being able to find ‘rc.exe’, ensure Windows Kits are added to your PATH. Example: C:Program Files (x86)Windows Kits8.0binx86
• For vague errors related to keywords, check if your versions of conda or Visual Studio are updated to their latest releases.
• Upon facing persistent issues, consider reinstalling in a new conda environment, to provide a clean slate.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Building Documentation and Running Tests

Building the user guide requires Intel® SDC installed along with a compatible version of Pandas and Sphinx. Check out the documentation processes in the README for detailed instructions on generating and viewing documentation.

Conclusion

Building and working with the Intel® Scalable Dataframe Compiler opens doors to efficient data processing and higher-performance analytics. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.