Increasing Chia farmer efficiency with Flexpool’s new ‘FlexFarmer’

You’ve probably seen my previous Chia posts, including how to build an efficient but sufficiently beefy plotter/farmer. As a long-time datacenter guy, I like building affordable, powerful servers.

Chia-specific cryptocurrency posts

However, as many farmers have found, if you plot and farm on the same machine, I/O can impact your farming performance, whether it’s disk I/O within the box, or writing off to a network share somewhere.

On top of that, you have to maintain a full Chia node, and optimally set up distributed harvesting with a somewhat complicated process. The full node currently requires about 13GB of local storage and frequent writes, so a Raspberry Pi or the like with an SD card is suboptimal. This also requires up to two days (for most computers) to sync the node initially, during which time you’re not at your best as a farmer.

Imagine if you could farm with the plots you have, using a tiny computer with very little CPU / RAM / storage requirements, without running and maintaining a full node, and saving 100W or more in the process.

Flexpool has just released their ‘FlexFarmer’ program which does just that.

Disclosure: I do work for Flexpool, but this post is based on my experience, not a press release or the pool admin’s expectations. Some of the testing I did was “on the clock” alongside support tasks, but I was not paid or required to write this post.

What does FlexFarmer do?

At a high level, FlexFarmer communicates with a node proxy on the pool server to communicate work and space, instead of requiring a local full node to operate. This means that anything that would require a full Chia node is handled on a powerful, resilient node at the Flexpool end.

You do still need to install the full Chia software to create your wallet. There will probably be ways around this in the future. You will also want a more formidable system for plotting, as the Raspberry Pi isn’t good for more than 1-2 plots a day.

FlexFarmer configuration file, redacted as appropriate

You download and install a binary called ‘FlexFarmer’ on your system, use a python script to get your farmer private key (used for validating partials and signing new blocks), and configure the farmer with a YAML file (above) with details like your farmer private key, your payout address, a system name (much better than the hexadecimal defaults), and where FlexFarmer should look for its plots.

Note that FlexFarmer only farms to flexpool. This has two considerations.

First, if you don’t pool with Flexpool, you can’t use FlexFarmer. This is easy to fix… just go to Flexpool’s Get Started page and join the pool.

Second, if your plot directories have plots from different PlotNFTs, you will get a lot of errors about the ones that aren’t compatible with Flexpool. You can ignore these errors, but I would recommend identifying the plots by pool contract and segregating them into different directories or folders (see the end of the post for more details).

For example, if you have two NFTs called Purple People-eater and Orange YougladIdidntsaybanana, you might have directories called /plots/purple and /plots/orange. In my case, the flexpool plots are in /<mountpoint>/flexpool, and solo plots in a “chia-final” directory at the same mountpoint.

Now two advantages you get with FlexFarmer, which were hinted at above:

First, you can name your farmer whatever you want, so you will easily know what farmer is doing what on the pool.

Chia farm workers listing

From this screen, I see that ‘chianuc1’ (which is a lot easier to remember than “c1dec8”) hasn’t been heard from lately. I go to look at it and see that it only supplies 6 plots, which is 60 points per day. Divide 1440 (minutes per day) by 60 (points per day) and I see on average I’ll get one point/partial every 24 minutes. With Flexpool’s 20 minute timeout, I will be “offline” a lot of the time until I get up to more plots.

A sample run, showing a plot in the process of copying, getting rejected. It will be picked up later.

Second, I can see on the screen (or in the optional logfile) when plots are added or removed. If the program checks while a plot is being written to its final destination, I’ll get that “Failed to initialize plot” error, but if I see other errors, I can investigate them pretty easily.

So how does this save me power and money and space?

I’m running FlexFarmer on three different machines, two beefy plotters and a slightly more salad-like NUC. The Lenovo x3550m5 (dual E5-2697v3, 256GB RAM, using various SSD and flash storage) ran at about 170 watts. Obviously the 1U server is bulky with a large footprint, noisy, generates heat, and isn’t that easy to move around.

I decided to try moving the farmer functionality off to a Rock64 single board computer that I had on my desk. It’s somewhere between a Raspberry Pi 3b and a Pi 4b, drawing about 9 watts at load. Add about 20 watts for a 7 port USB 3.0 hub (since the SBCs are usually limited to a couple of USB ports), and we’ve brought that 170 watts down to under 30.

So we’ve gone from 29x17x1.75 inches and about 35-40 pounds, dual power supplies using 170 watts, a full-ish OS load (on a 300GB disk in this case), and being very loud and warm…

To 4.5×3.25×1.5 inches (in a larger-than-absolutely-necessary case), using 9 watts or less powered from a 5V USB charger booting from an 8GB MicroSD card, generating no sound and no noticeable heat.

The drives are still going to use some power, probably 10 watts on average per drive. So for 56TB of plots, my FlexFarmer Pi-class farmer will use under 100 watts, compared to just under 300 watts for the full server.

Sure, I don’t want to plot on a single board computer like this, so the Lenovo and the Dell T7910 will still be useful when I have new disks to fill. But for the ongoing farming efforts, the work that actually brings in some Chia coin, I can run that platform with lower space and power without taking up a full computer.

Is FlexFarmer for everyone?

Not necessarily. If you’re following Bram Cohen’s “use what you have” minimalist strategy, you are probably fine just running the Chia Network software on your PC with internal or external storage.

If you’re plotting on all your farmers, you probably don’t need a second farmer, although you might look into isolating or minimizing I/O on the farmer for better farming results.

And if you’re dedicated to solo farming, or a Chia pool other than Flexpool, it’s not really an option.

But if you have farmer systems with lots of drives attached that aren’t plotting, or if your farmer is already low horsepower, or if you’re looking to farm from work or your partner’s house (with permission of course) without a full-on PC involved, or if you just want to try out farming efficiently from a 1GB single board computer, FlexFarmer is definitely worth a look.

How do I get started?

First, you need a computer to run FlexFarmer on. It will do fine on at least a Raspberry Pi 3b or pretty much any PC from the last 10 years that runs current Linux distros (Windows support is coming).

If you don’t have a Pi or cheap computer around, here are a couple of easy starters.

  • CanaKit Raspberry Pi 4 Starter Kit – 2GB RAM for $90 (or go with 4GB kit for $100, or 8GB kit for $120). This has everything you need, from the board to power supply to preloaded Micro SD card.
  • You can also go with the Pi 3 basic kit – 1GB RAM for $55.
  • If you’re more comfortable with an x86/x64 computer, Beelink T4 is available for about $120. I have a couple of Beelink systems but haven’t tried them out yet. They come with Windows 10, but you should be able to replace it with Ubuntu if you prefer, and they will likely use well under 40 watts of power.
  • If you have more than 2 external drives, you will probably want a powered USB 3.0 hub. USB 2.0 may be enough, but the cost difference is not that big, so consider something like the Anker 7-port 36W powered hub for $50. Be careful with higher port count hubs, as most of the sub-$60 10-port or larger hubs have several charge-only ports. Great for powering a compact Ethernet hub or charging your phone, but useless for connecting hard drives.

Next, you need the FlexFarmer software. Download it here.

Follow the instructions in the download file once you’ve set up the computer and updated the software on it. You should be up and running in minutes from there.

See another piece on this fun new plotter from The Chia Plot blog.


A bonus: How to discover which plots belong to which Plot NFTs.

Most people can stop reading now.

This is a pipeline that evolved from a Reddit post, through a couple of gurus, and then got defiled by my lazy sysadmin enhancements.

curl --insecure --cert ~/.chia/mainnet/config/ssl/harvester/private_harvester.crt --key ~/.chia/mainnet/config/ssl/harvester/private_harvester.key -d '{}' -H "Content-Type: application/json" -X POST https://localhost:8560/get_plots | python -m json.tool | grep -B5 '"pool_public_key": null' | grep -o '/.*\.plot'

If you run this command on a Chia node with the full software load (not just FlexFarmer), it will spit out a list of all PlotNFT associated plots. This will let you segregate OG plots from modern ones.

If you cut the command off after json.tool:

curl --insecure --cert ~/.chia/mainnet/config/ssl/harvester/private_harvester.crt --key ~/.chia/mainnet/config/ssl/harvester/private_harvester.key -d '{}' -H "Content-Type: application/json" -X POST https://localhost:8560/get_plots | python -m json.tool

You will get a JSON structure for each plot containing useful details.

    {
        "file_size": 108834556275,
        "filename": "FILENAME.plot",
        "plot-seed": "PLOT_SEED",
        "plot_id": "PLOT_ID",
        "plot_public_key": "PLOT_PUBLIC_KEY",
        "pool_contract_puzzle_hash": "POOL_HASH",
        "pool_public_key": null,
        "size": 32,
        "time_modified": 1626064238.0
    },

The “pool_contract_puzzle_hash” is what you’re looking for to segregate multiple hashes. Map that to what you find from “chia plotnft show” – look for:

Wallet id 2:
Current state: FARMING_TO_POOL
Current state from block height: 648421
Launcher ID: LAUNCHER_ID
Target address (not for plotting): TARGET_ADDRESS
Number of plots: 16
Owner public key: OWNER_PUBLIC_KEY
Pool contract address (use ONLY for plotting - do not send money to this address): POOL_CONTRACT_ADDRESS
Current pool URL: https://xch-us-west.flexpool.io
Current difficulty: 1
Points balance: 0
Percent Successful Points (24h): 0.00%
Relative lock height: 100 blocks
Payout instructions (pool will pay to this address): PAYOUT_ADDRESS

Use https://www.chiaexplorer.com/tools/address-puzzlehash-converter to convert the POOL_CONTRACT_ADDRESS (starting with xch) to a puzzle hash (starting with 0x).

Now you can run the above command with a creative grep (or whatever method you prefer) and identify the filenames for any plots associated with a given PlotNFT. For example, here’s the messy method I use.

curl --insecure --cert ~/.chia/mainnet/config/ssl/harvester/private_harvester.crt --key ~/.chia/mainnet/config/ssl/harvester/private_harvester.key -d '{}' -H "Content-Type: application/json" -X POST https://localhost:8560/get_plots | python -m json.tool   | egrep "{|filename|pool_contract_puzzle_hash" | fgrep -C2 "0xPOOL_HASH" | fgrep filename | cut -d\" -f4

The output is a list of filenames which you can then move, archive, rsync, etc to a plotnft-specific directory. I’m sure there’s a more graceful way, and I’d love suggestions in the comments, but this works for me.

4 thoughts on “Increasing Chia farmer efficiency with Flexpool’s new ‘FlexFarmer’

  1. Pingback: FlexPool releases new light weight Chia farming software FlexFarmer - The Chia Plot

  2. I created all my NFT portable plots as a solo farmer and I am still solo farming using MadMax to plot my NFT’s. Are you saying that when I move my NFT plots to FlexPool and decide to try their FlexFarmer that I will see a bunch of errors because they were all created with my onw self pooling plot contract address? I will never create a NFT plot with any address other than my own self pooling contract address.

    Like

    • Pooling to Flexpool works the same way as any pool using the Chia pooling protocols works, whether you use Flexfarmer or Chia Network 1.2.x or later.

      You create a PlotNFT. It generates a P2 singleton address.

      You create plots with that PlotNFT’s “P2 singleton address (pool contract address for plotting)” and your Farmer Public Key. These are the -c and -f parameters to chia_plot.

      chia_plot -n -r -t -2 -d -c -f

      If you want to pool to a pool, you join the PlotNFT to that pool. If you don’t join the PlotNFT to a pool, you have no way of farming to that pool.

      Like

      • And of course WordPress squashed the formatting on that. But in short, you would join Flexpool with your PlotNFT the same way you would join Spacepool or xchpool or 21chia or whatever other pool.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.