• 0 Posts
  • 10 Comments
Joined 3 years ago
cake
Cake day: September 10th, 2023

help-circle
  • MCP in general:

    1. MCP is very verbose (json tool declarations)
    2. Every tool incurs a prefix cost, which can be in the thousands of tokens across all tools
    3. Tool outputs are usually dumped straight into the prompt. Sometimes they are truncated and dumped into a file with a warning to the LLM, but usually after they reach thousands of tokens
    4. Changing any tools breaks your prefix cache.
    5. Tool discovery can help with 1 a bit, 2 a lot, and 4 a lot, but isn’t a cure-all. It does have its own (much smaller) token overhead too.

    CLI in comparison:

    1. Fairly concise, by nature - short names for frequently used commands, spaces as implicit delimiters, implied outputs.
    2. Tools are generally implied to be present. A short “this environment contains x,y,z,” system prompt can be very effective. Very minimal prefix cost.
    3. Tool outputs can be written to a file directly, formatted, head/tailed, grepped, or used in any other way.
    4. Doesn’t break the prefix cache, but can “confuse” the LLM.
    5. Tool discovery is implicit in the system. This does have its own overhead, but for many things is implied - ls, find, grep, sed, cat - usually an llm will just assume they’re present, and use them to trim down tool outputs to only the required context. In f, I also added search functionality to avoid parsing ‘help’ messages.

    CLI does of course have its own downsides, I think the biggest being security. Give an LLM an MCP server, and all it can do is what that MCP server supports. Give an LLM a Turing complete shell, and it can do anything.

    MCP also tends to be harder to set up and harder still to fix when it doesn’t work. I’ve watched Claude fall back to curl commands when a connector doesn’t work many times. I’ve never seen it try curl commands, fail, and then try to set up MCP.

    I think this is all fairly domain dependent, but I think my domain (software development and general agenetic assistance) is very common. If you’re using an LLM in a product that you’re selling with a fixed range of features, MCP is probably the way to go. If you’re using an LLM for dynamic functionality with expanding capabilities, I think CLI might fare better.

    My main reasoning for all of this is flexibility. In the history of computing, a program has always had a fixed output for a fixed input. LLMs are at the point where they can handle what is effectively an arbitrary input and give an arbitrary output. Giving an LLM a fixed set of tools negates that newfound ability to process the unknown. It’s like giving someone a sledgehammer to bust rocks, but tiring one hand behind their back; the work might get done, but not as efficiently.


  • I mean, I haven’t really started discussing it outside IRL friends, so not surprised you haven’t heard of it 😅

    In essence, the problem is twofold:

    1. LLMs tend to use CLIs more effectively than MCP, at least in the domains I use. However, installing all the CLI tools and configuring them can be cumbersome and not easily repeated. CLIs also cannot be modified, and are often quite verbose.
    2. LLMs will repeat very similar commands over and over with just a few tweaks. This often happens when there is a gap in a tool or poor documentation.

    So fcli is intended to be an easily extended, configured, and authenticated CLI. The “one CLI” to do everything your LLM needs. Ideally, you set up a personal repo to store ‘ucks’ (f packages) in, and you set up credentials for that repo so your LLM can push any new ucks to it. You can then share that repo with other users if they could also benefit from any of that functionality.

    As you go along in a session, when you notice the LLM repeatedly running similar verbose commands, you can have it create an uck for it instead and save those tokens.

    Likewise, if, say, an MCP server doesn’t have full functionality, you can have your LLM import the SDK from npm into an uck and write a basic fcli wrapper around it to start using.

    The other nice thing is that it makes your tool configuration repeatable between LLMs and harnesses. You don’t need to set up your MCP servers in both Claude and Codex and Pi, just in f and tell them to use f.

    There’s also a creds uck to make your secrets more portable and (a bit) safer.

    In general, I’m trying to model it a bit like pi.dev, but as part of the tooling instead of the harness itself. I think they work quite well together :)

    Interesting paper comparing MCP and CLI


  • Running Qwen 3.8 27B dense on some R9700s, and Qwen 3.8 flash next on dgx sparks. I use it for code gen outside work, including an automated Dev pipeline that rubs on its own. When I did the math comparing to token API pricing of Opus 4.8, it covered its own costs in a few months. With the sparks I get about 40tg/s, 1500pp/s, but can get concurrency up to 5 stably. With the R9700s, I get ~100tg/s, 3000pp/s and can also get up to 5 concurrency.

    I access it over wireguard to the network that it is on, and use termux to ssh into a build machine for dev over CLI using https://pi.dev/

    I also started building https://fcli.dev/ to make integrations easier; the idea being that LLMs should just implement simple reusable functions instead of repeatedly making complex curl requests to integrate with a service.

    I also use LLMs heavily for work; Qwen 3.8 seems almost on par to Opus 4.8 IMO. I also don’t have to worry about token costs, and it often responds faster than Claude would because there usually isn’t a queue on my machines. At this point, I wouldn’t complain if my work provided me 2x B6000 instead of a Claude subscription; I’d still be able to get all my work done.


  • If I’m bleeding any more than a drop or two of blood, I’m reaching for the closest remotely clean and absorbent material while I go find a real bandage and some neosporin. 99% of the time that is a paper towel since they’re always around and handy. Never got an infection that way, and I think it’s a perfectly reasonable reaction when you or someone else is bleeding…

    Edit: and to be clear I don’t think paper towels are sterile, but neither is the air that I’m running through to go get a bandage; might as well not get blood everywhere while I’m at it.





  • Well then very little of what I said actually applies!

    Unless you know the hours on a drive, you might get brand new ones, or you might get ones with 50k hours on them. They may also be from the same batch, which isn’t ideal for data durability. If you’re ok with all that, then go for it. I generally don’t buy used drives because I don’t want to take the additional risk.

    I’d be surprised if you can’t find a better deal on used spinning rust though… the shipping alone is probably half the value on a good chunk of sales from SmS.


  • I get that, that was also something I used to like about old servers, but let me float a few of the things that I’ve come to realize through my home-lab career to you:

    • Raid is perfectly feasible in consumer hardware. If your motherboard doesn’t have enough SATA ports, you can always get an HBA or a JBOD to support for more disks. There’s really no good reason (that I have heard of) for hardware raid today. Just remember raid is not a backup :)
    • There are consumer ATX PSUs with redundancy. However, the only reason for PSU redundancy is when you cannot tolerate downtime due to a PSU or UPS failure, and that redundancy might save you a few hours of uptime over 10+ years in comparison to a non-redundant consumer PSU that you can go out and buy if it fails. When was the last time you had a (reputable) PSU fail on you? What kind of uptime are you targeting? If you don’t have an answer for that, 99% is very easy to reach even on consumer gear, and is a strong indicator that you don’t need enterprise levels of redundancy. 99% is literally 3 days of downtime per year. Also keep in mind that redundant PSUs are just going to gobble more power and increase operating costs.
    • KVM features - this was the big one for me. I wanted to be able to perform out-of-band remote maintenance on my servers. Then I took a leap and got a Sipeed NanoKVM, and I haven’t looked back. there are plenty of them out there - PiKVM is another reputable one. When buying old enterprise servers, you often have to pay for the remote management license, and that is just another added cost. Not to mention that they lose support pretty quickly, and you end up running out of date software on one of your most critical interfaces to the machine. A NanoKVM, PiKVM, and others aren’t built into the machine, so they continue to be supported for much longer.

    One other thing that I’ll mention and you probably already know - enterprise servers are LOUD - even just a single one can literally sound like a jet engine. That’s not a hyperbolae. If this is your first one, don’t underestimate it. I had my servers in the basement with decent insulation, I used IPMI to throttle the fans back to 10%, and I could still hear the whine on my first floor when everything is quiet. If you end up having to turn down the fans due to noise, you’re going to start having heat issues, and then you’re losing out on performance and shortening component lifespan. Noise-proofing a server is non-trivial - you have to allow air flow still, and where there’s air flow, there’s a path for noise too. My current setups all have 120mm and 140mm fans, and I can barely hear them when I’m working right next to them. My 3D printers are the loud ones in the basement now!


  • Yeah, they’re legit. Bought a few servers from them over the years. No major issues, packing was good, reasonable ship time.

    Had one case where they sent a different NIC than what was listed. They just shipped me the correct one and told me not to bother sending the old one back.

    Stopped buying from them though because I prefer off-the-shelf modern consumer hardware nowadays. The real cost is always power consumption, and I prefer to shell out more money up front in exchange for huge savings on power usage down the line. I can always run over to microcenter and replace a part same-day as opposed to ordering it online and hoping it comes soon.

    If you’re a home-labber, I’d strongly suggest doing the same. Some of those old enterprise servers just gobble power for not that much compute relative to current day consumer machines.

    If I was still buying older servers though, I’d probably be looking at their prices.

    What are you considering buying?