onfocus

Rather than needing tens of thousands of machines and millions of dollars to train a new model, an existing model can now be customized on a mid-priced laptop in a few hours. This fosters rapid innovation.

Nice summary of how innovation in AI might move out of the largest few companies.

Coupled with the Writers Guild strike and the arguably reckless pace at which companies are willing to adopt a mostly unproven, experimental, and demonstrably harmful technology, the world seems to be falling headfirst into a labor struggle the likes of which it hasn’t seen in quite a while.

The answers here are to vote for labor-friendly politicians and unionize. Good evergreen advice but also frustratingly vague for a specific looming threat.

That’s because there is no actual precedent for saying that scraping data to train an AI is fair use; all of these companies are relying on ancient internet law cases that allowed search engines and social media platforms to exist in the first place. It’s messy, and it feels like all of those decisions are up for grabs in what promises to be a decade of litigation.

The current round of language and image model speculation is based on the premise that using any public data for training is fair use not a massive copyright violation.

The Post’s analysis suggests more legal challenges may be on the way: The copyright symbol — which denotes a work registered as intellectual property — appears more than 200 million times in the C4 data set.

This humble website is included in the C4 corpus. You can use this tool to see if your copyright has also been violated.

If we as technologists want to help the broader public understand these AI systems, both their opportunities and challenges, then we need to speak in plain language.

I do think we need to go back to the beginning and just say “ChatGPT lies”.

I’m guilty of trying to find the perfect word with the correct nuance or shade of meaning to describe a situation. Sometimes that impulse works against clarity.

36% of another sample of 480 researchers (in a survey targeting the language-specific venue ACL) agreed that “It is plausible that decisions made by AI or machine learning systems could cause a catastrophe this century that is at least as bad as an all-out nuclear war” (Michael et al., 2022).

So we've got that going for us. LLMs strategically manipulating people into acquiring power sure sounds like a serious flaw in the software. A bit more information and context at the unfortunately named NYU Alignment Research Group. (ARG? Seriously?!)

I like to think of language models like ChatGPT as a calculator for words.

This is reflected in their name: a “language model” implies that they are tools for working with language. That’s what they’ve been trained to do, and it’s language manipulation where they truly excel.

Nice framing for thinking about the best ways to use LLMs.