BlockchainSpider is a powerful tool designed to crawl and collect data from public blockchain networks. It provides insights into transaction subgraphs, address labels, and block data which are essential for analyzing transaction patterns and addressing potential illegal activities.

Getting Started

To get started with BlockchainSpider, you’ll need to clone the repository and install the necessary dependencies. Here’s a friendly guide to help you through the process.

Install

Run the following command to clone the BlockchainSpider repository:

git clone https://github.com/wuzhy1ng/BlockchainSpider.git

Next, install the required dependencies using:

pip install -r requirements.txt

Crawling a Transaction Subgraph

Let’s demonstrate how to crawl a transaction subgraph, specifically focusing on the KuCoin hacker on Ethereum. This process will allow you to trace the illegal funds associated with the hacker.

Execute the following command:

scrapy crawl txs.eth.ttr -a source=0xeb31973e0febf3e3d7058234a5ebbae1ab4b8c23

Upon completion, you’ll find the transaction data saved in the file data0xeb3...c23.csv. Import this transaction data along with the addresses’ importance into Gephi for further analysis.

Collecting Label Data

Now, let’s collect labeled addresses from the OFAC sanctions list. This is crucial for identifying potentially malicious entities.

Run the following command:

scrapy crawl labels.ofac

The label data can be found in the datalabels.ofac file, where each row is a JSON object containing essential details such as entity names and email addresses. Remember to indicate your source when utilizing these labeled addresses.

Collecting Transaction Data

Let’s shift gears and collect transaction data on Ethereum. You can continuously gather transactions starting from block number 19000000 to the latest block.

Run the command as follows:

scrapy crawl trans.block.evm -a start_blk=19000000 -a providers=https://freerpc.merkle.io

The transaction metadata will be saved in the data folder, with:

• BlockItem.csv storing block metadata like minter and timestamp
• TransactionItem.csv bearing external transactions of blocks

Additionally, BlockchainSpider supports collecting transaction receipts, logs, and token transfers. It also has the capability to gather block data from EVM-compatible chains, including BNBChain and Polygon. For more details, refer to our documentation.

Troubleshooting Tips

• Make sure you have Python installed and that you’re using the correct version required for this project.
• If commands fail, ensure all dependencies are installed correctly.
• Check the network connection; sometimes, crawling requires a stable internet connection.
• For further assistance, consult the comprehensive documentation.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Final Thoughts

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

By following this guide, you now have the foundation to utilize BlockchainSpider effectively for your blockchain data collection needs!