I’ve been using normal Nix for a couple of years now. I didn’t even know there were drop in replacements! I’m looking at changing to Lix now.

score 1 · canonical

• Wholesale pattern-matchable copying of code without attribution, whether by human or machine, is inarguably illegal. This has nothing to do with community.
• I accept that non pattern-matchable influence between programmers and codebases is legal. This then does indeed become a question of community. So if you accept that it is moral or at least okay, then you must therefore also believe that it is wrong for OpenAI and Anthropic to restrict their models from reverse engineering, cloning copyrighted software, binary analysis, etc?
• My actual point is about how the sentiment of mitchellh’s comment so easily also reflects the feeling of many open source creators. This is not about legality. If one believes themselves justified in expressing such feeling towards trivial dismissal of their AI-assisted work then I would hope that one is also able to accept those exact same feelings from those affected by LLM output’s lack of attribution.
• Why would somebody mark my comment as spam?

score 1 · canonical

I wonder what your thoughts are on the legality of LLM output? I’m thinking of papers like this one, https://arxiv.org/html/2408.02487v3, that find up to 2% of LLM output is “strikingly similar to existing open-source implementations”.

It’s hugely insulting to me and others who have worked hard to produce [high quality libraries] with trivialized dismissals of years of a humans actual craft and labor.

The reason I ask is because I was struck by how easily that quote also reflects the sentiment held by many open source creators. Trivial dismissal is a perfect way of describing the feeling of those humans that have worked hard to produce high quality software over years of actual craft and labour, only for those efforts to benefit the world without recognition.

You invited the challenge of scrutinising your knowledge of the libghostty-vt codebase. I suspect you aren’t aware of the possible 2% of the LLM-generated portion of the codebase that bares a striking resemblance to existing open source implementations. Sure, we all used to copy and paste bits and pieces from here and there in the before times anyway, so maybe that doesn’t seem like a big issue. But what if certain non-pattern matchable aspects of libghossty-vt bear a striking resemblance to solutions from, let’s say, that very same cohort of non-affluent teenagers you mention? Except this time there is no trace of provenance.

I don’t doubt for a second that you would be among the first to provide attribution if there were a practical way to do so. And I don’t believe that there is any meaningful way to describe Ghostty as vibecoded. I am just struck by how applicable the overall sentiment of your message is to the very body of weights and biases upon whose shoulders AI-assistance stands.

score 1 · canonical

Ohh those are some interesting ideas! Thanks. I look forward to that blog post.

score 1 · canonical

This might be a tangent to the main conversation, but I was wondering about your AI usage policy where you write:

As a coder, I do not think there is any ethical issue with using AI to write code (outside of environmental and societal impacts, see below)

Where do you stand on the licensing concerns of LLM output? I’m thinking specifically about papers like https://arxiv.org/html/2408.02487v3 that conclude:

Using LiCoEval, we evaluate 14 popular LLMs, finding that even top-performing LLMs produce a non-negligible proportion (0.88% to 2.01%) of code strikingly similar to existing open-source implementations. Notably, most LLMs fail to provide accurate license information, particularly for code under copyleft licenses.

From your other views, I feel like you probably think this is unethical, but it’s not clear from that line I quoted in your AI usage policy. Maybe you have an interesting take?

score 1 · canonical

Shame on anybody marking this off-topic. I’m a man in tech, and men so often get in the way of actually working.

• I started a new contract. CEO calls meeting with me and the team to have a high level technical discussion. He monologues for 50 minutes. I address him as a tech equal, reminding him that good communication is key to good software. I’m seen not to be a “team player”.
• Remote colleague has unhelpfully strong opinions on Slack about technical decisions, but won’t have video meetings. We have to put up with him because he’s good value for money.
• Old friend of the company and occasionally contractor joins project, rewrites large portions of the codebase rather than communicating with the existing team.

I’m not saying this to make it look like it’s just as hard for men as it is for women in tech. I’m just pointing out that the matter of woman in tech is profoundly on-topic. It’s not merely some, “yeah wouldn’t that be nice to fix”, but rather a technical concern at the very heart of working at the coalface in this industry. Software is literally worse and harder to make because of male culture.

score 2 · canonical

I don’t think it’s the “in passing” bit you disagree with. But rather the bit about Anthropic’s agenda.

I can’t believe that I need to spell out that the entire reason the mafia are talking about coffee, is for the cover of mentioning things in passing. If you pushed them about their meaning, they’d just throw up their hands and say, “Hey, hey, I’m just saying the coffee is nice. Don’t be so paranoid.”

score 2 · canonical

I’m not sure the post in question merely mentioned LLMs in passing:

Speaking of swag, I am 100% making stickers with the srand48() seeds, and t-shirts for the Anthropic team

Anthropic, like Google, is in the business of marketing first, and only tech secondarily. The sheer fact that the NIST seeds are being talked about is part of, what could well be, a slop narrative. Namely that Anthropic is “pioneering” AI.

It would not at all surprise me if this whole thing comes from a prompt, like some manager at Anthropic just typed: “think of cool things that we can do with our in-house models that keep us in the news cycle. then do the ones that seem most possible. only report back on ones that are successful”.

To what larger narrative does the NIST seeds story belong? For example, when I read the TEE attacks survey story currently on the frontpage I have the sense that it’s performing a public service to the secure community as whole. But the only thing I take from the NIST seeds story is an implicitly threatening reminder that Anthropic has enormous unchecked powers. It’s like the mafia calling into their local coffee shop, casually mentioning how delicious the coffee is, and how it’d be shame if anything bad were to happen to the building.

The vibecoding tag was deserved.

score 22 · canonical

Who made this? And why haven’t they put their name to it on the website?

score 1 · canonical

Aleph Alpha, along with other notable AI companies like Anthropic and OpenAI, are signatories to the EU’s General-Purpose AI Code Of Practice[1]. In Section 1.4 they address LLM output:

Measure 1.4, Mitigate the risk of copyright-infringing outputs

(1) In order to mitigate the risk that a downstream AI system, into which a general-purpose AI model is integrated, generates output that may infringe rights in works or other subject matter protected by Union law on copyright or related rights, Signatories commit:

• a) to implement appropriate and proportionate technical safeguards to prevent their models from generating outputs that reproduce training content protected by Union law on copyright and related rights in an infringing manner, and

• b) to prohibit copyright-infringing uses of a model in their acceptable use policy, terms and conditions, or other equivalent documents, or in case of general-purpose AI models released under free and open source licenses to alert users to the prohibition of copyright infringing uses of the model in the documentation accompanying the model without prejudice to the free and open source nature of the license.

In the paper, LiCoEval: Evaluating LLMs on License Compliance in Code Generation[2], they conclude that:

Using LiCoEval, we evaluate 14 popular LLMs, finding that even top-performing LLMs produce a non-negligible proportion (0.88% to 2.01%) of code strikingly similar to existing open-source implementations. Notably, most LLMs fail to provide accurate license information, particularly for code under copyleft licenses.

I can’t vouch for the percentages, but I can say I’ve never seen or heard of an LLM providing licensing information for the code it generates. I know it’s just a code of practice and not law, but I don’t think anybody is even attempting to honour Section 1.4.

1. https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai
2. https://arxiv.org/html/2408.02487v3

score 2 · canonical

AI itself is fruit of the poisoned tree.

If you pay somebody else to care for your children, you don’t get to complain if you encounter issues integrating with your children’s lives. Are you part of the family or not? Ethics isn’t one of many possible stances you can take to having a family, it is the very basis of relating to other human beings. If you choose to ignore humans in endeavours that fundamentally involve humans, then you are the problem, not the humans.

Do parents that pay for carers have a “dilemma” about being involved with their often cold and standoffish children? No. Or rather it’s disingenuous to frame it as a dilemma. The blame is squarely at the feet of the parents.

Sure, I get it, we’re so much more productive when we get a third-party to care on our behalf. But that comes at a price, one of which is losing our voice.

Open source is our child. It doesn’t demand anything other than that we acknowledge it. Copy it, sell it, change it, make it your own. All it asks is that you attribute your sources. It’s a profoundly simply contract. If you can’t even achieve that most basic of tasks, then it’s hardly surprising that you’ll be troubled by more challenging tasks further down the line.

AI does not care for the welfare of our ecosystem. That is the security issue. It is a logistical impossibility to steer a thriving and successful community towards integrating with a technology that undermines its core principles. As we should all know, attacking software takes many forms, it’s not just about exploiting bugs, it can also be about denial of service and social engineering. So to even suggest that anti-AI policies create a dilemma for pro-AI users finding bugs and vulnerabilities is a misleading framing of the reality of the situation.

You cannot both care that you find bugs in open source software and not care about the provenance of your LLM output. The consequences of not caring are entirely on you, not the ecosystem.

score 1 · canonical