armcat
I keep seeing these "sovereign" LMs time and time again.
In Sweden we had GPT-SW3
(https://www.ai.se/en/project/gpt-sw3) and same story
there. Instead of burning money on "sovereign" claims,
national research labs should instead focus on building on
top of solid baselines (like Qwen/Kimi) and finetuning
frontier models with real agentic utility that can be
applied across actual use cases and can be widely used by
its people, basically for free. Nations should mirror what
Cursor has done with Composer 2.5 for example.
|
> appplication
Disagree, it's in the country's best interest to
facilitate internal expertise on the full stack and
own their "supply chain" so to speak and fight brain
drain. The outcome isn't just the model, it's the
expertise. Otherwise all their smartest folks will
depart for countries where LLM development is
strongest.
|
> > bradhe
Got it, so every country should focus on having a
mediocre-at-best AI strategy by refusing to work
together? Surely this will create a better future
instead of pooling resources.This
vaguely-nationalist world view around tech that's
emerging in Europe is dangerous, man.On the brain
drain problem in particular, one way to ensure
talent sticks around is to create a good
environment for people to do their best work. In
much of Europe, getting bureaucracy out of the way
and encouraging real investment would go a long
way. People leave because they can make more money
and they want to be surrounded by the best people.
People would trade some of that off to stick
around their home countries, however if you go to
California and talk to folks from e.g. NL or DE
working on this stuff, they have a lot to say
about innovation and working culture back home.
|
> > > Certhas
This "vaguely nationalistic world view around
tech" is a direct consequence of the US
government weaponizing its leading tech firms
reach into the EU for nationalistic
purposes.The EU was build on the principles of
collaboration, to overcome the nationalistic
impulses. Free trade and free movement of
people, no need for everyone to replicate what
everyone else is doing. Preventing individual
nation states from favoring and supporting
their home grown firms over other EU firms is
a central legal principle.But this only works
if it is reciprocal. And ideally if the
partners are roughly on the same level. When
you force developing nations in trade deals to
not protect local firms, you are also
preventing them from moving up the value chain
and locking them into the position of "raw
material providers".When trading with China we
know that China has absolutely no qualms
supporting its strategic industries with a
truck load of subsidies. And it is preventing
foreign firms from investing and selling on
the domestic market.For the longest time the
US was considered a safe partner though. Sure,
plenty of disagreements in the details, but in
principle someone whom you can rely on. That
idea has been decisevly dismantled over the
last 10ish years. The US unilaterally cut of
the EMails of EU courts. The US has
unilaterally decided that EU partners cannot
use Fable/Mythos.The only reasonable reaction
is to make sure that the EU can maintain and
create its own critical infrastructure.
|
> > > > nradov
Let's not overstate the case. The EU was
built to keep France and Germany from
getting into another war, and it has been
successful at that. All the other
"principles" are just window dressing.
|
> > > > > mvc
Isn't it up to Europeans to define the
purpose of the EU? What's your claim
to having the right to define it?
|
> > > > > > nradov
I'm not defining anything. I'm
just explaining why the key
decision makers actually created
the EU. The statements made for
public consumption shouldn't be
taken too seriously.
|
> > > > > DiogenesKynikos
You're vastly underestimating the
economic importance of EU integration.
|
> > > > > > nradov
Is that why the EU economic growth
rate is so stagnant?
|
> > > > > > spacemanspiffii
Simplistic reply without
substance. The EU economic growth
is influenced by much more than
just the integration. It can be
stagnant not because but despite
the integration. It could also be
the case that EU integration was
an attempt to improve economy but
it didn't work out.
|
> > > > > > nradov
So no evidence of economic
importance.
|
> > > > themgt
This "vaguely nationalistic world view
around tech" is a direct consequence of
the US government ... The EU was build on
the principles of collaboration ... But
this only works if it is reciprocal.This
sounds great but doesn't really make any
sense. What skeptics are saying is there
should be a pan-EU effort to build
frontier models, rather than one-off toy
models built by each country as a
box-ticking exercise."We built the EU as a
powerful supranational organization, so
logically the EU's response to a great
power challenge in the realm that may well
define the 21st century is gonna be to
shatter its efforts into 27 useless
pieces, because Trump bad" is just
absolutely ridiculous and will not lead to
anything good. Be the change you want to
see, etc.
|
> > > > > Certhas
Look I agree with all you said. The EU
should do this but the EU is also
structurally bad at large concerted
efforts exactly because it is a
collaboration mechanism, not a super
state.
|
> > > > > > akie
Whatever topic comes up on this
board, if the EU is mentioned
people go a bit cray-cray.If an
individual country trains a
language model, it's not ambitious
enough. If we try to do one for
the whole EU they will say it
takes too long (you need to get
all countries on board, you see).
If the EU announces it by
executive decision it's a
dictatorship and government driven
economic intervention, if a EU
company does it they'll say it's
not good enough.You can't win with
these people. In my opinion, you
shouldn't even try to convince
them.
|
> > > > trhway
>This "vaguely nationalistic world view
around tech" is a direct consequence of
the US government weaponizing its leading
tech firms reach into the EU for
nationalistic purposes.And in response NL
should weaponize ASML for example. Then
both sides would naturally back down.
Specialization is the most efficient way
of developing tech civilization.Whereis
everybody building their own mediocre
versions would be repeating Russia in its
attempt to make its own national messenger
- a lot of government money sunk, yet
people are still using Telegram/etc.
|
> > > > > saidnooneever
Of NL weaponizes ASML they will just
take their IP and leave NL and then
they have nothing. They have already
pushed a lot of things out of NL and
should invest in leveraging the
position without weaponizing it.
|
> > > > > > enaaem
If ASML is just IP then everyone
would have already copied it. They
have an extremely deep and high
tech supply chain that is nearly
impossible to copy.
|
> > > > > dijksterhuis
ASML wiki for those like me who were
confused
https://en.wikipedia.org/wiki/ASML(des
pite the name, i am not dutch)
|
> > > > > > jaapz
I am quite interested to hear how
you would pronounce your name,
dike-ster-house (assuming you live
in the US)?
|
> > > > > > dijksterhuis
> dike-ster-houseyep, basically.
3rd one down
https://www.howtopronounce.com/dij
ksterhuisthe story behind it: i
was looking for a name to write
music under. and i was reading
academic papers about subliminal
messaging in music and stuff. i
came across a paper where they
seemed to be absolutely ripping
into this "dijksterhuis" guy's
previous paper. felt bad for him.
felt like he was an underdog. so i
ran with that name.
|
> > > > > > fransje26
> (despite the name, i am not
dutch)Some close or far ancestor
was though. :'-)
|
> > > Gud
Why would it have to be mediocre?Both
Netherlands and Sweden produce highly
competent researchers. Per capita on par with
any other location on the planet, including
California.This will be good for Europe as a
whole and also the planet.
|
> > > > colordrops
Sure, but there is still a more dense
talent pool with SV companies, and
training frontier models requires massive
amounts of capital. Are Netherlands and
Sweden prepared to invest 10s of billions?
|
> > > > > varjag
My impression is the talent pool for
core model development in SV isn't
huge and is already grabbed by the
usual suspects. Everyone else sloshing
in the vague broader AIsphere are
application bros whose value in the
supply chain is minimal.
|
> > > > > Gud
Honestly I have a strong suspicion
that "the talent pool" will shift
towards Europe and China and I believe
it's already happening.And yes, I do
believe Europe will invest in this
technology.
|
> > > > kortilla
Producing local researchers isn't enough.
If you want a competitive model you need
to bring the best researchers from all
over the world. That means lots of
investment that hasn't been common in
Europe.
|
> > > > tcp_handshaker
You missed the part where they got 13
million EUR , 3 years ago! and dont have a
release yet....Definitely doing it at
European pace...they will have release two
when AGI is around...
|
> > > > vitalyan123
>and also the planet.What a glorious day
for Canada and, therefore, the world.
|
> > > > > Gud
I don't think it's good that the USA
has so much influence due to their
dominance in information technology.It
was only nominally democratic, now
it's a total shit show.
|
> > > > > > budsniffer952
>total shit showAre there people
legitimately brain-rotted such
that they believe stuff like
this?Outside of the constant flow
of "hey guys, look at the latest
dumb thing Trump did" coming from
entertainment outfits posing as
news, what are we even talking
about? Life goes on as normal for
the Americans. You are way too
online.
|
> > > > > > Gud
Yes, the current administration
has been in power for less than
two years so obviously the "day to
day life" hasn't changed much for
the average American (except
everything has gotten more
expensive).The US is no longer
looked at as a superpower, it just
surrendered to a regional power
and will pay $300B USD in
tribute.But for Trump and his
cronies life certainly has
improved.But the damage is done
and we are witnessing the decline
and destruction of the worlds
greatest Empire of all time.
|
> > > > > > trollbridge
Nitpick: it's 6 years.
|
> > > > > > Gud
The Republican Party looked very
different during Trumps first term
|
> > > > > > mannanj
The us stopped being looked at as
a superpower as a brain rotted
Alzheimer's patients president
rose to power on the campaign
"don't let Trump win" and then did
little productive except stoke
wars and pardon his son and his
friends from crimes.
|
> > > > > > Gud
I'm assuming you're talking about
Biden, and no, not really. Still
considered a superpower.Up until
the second Trump presidency, most
of us outside the USA had
basically the same opinion as we
had before. Pretty messed up
politics , but strong economy and
strong military.The first minor
dip was with Trumps first
presidency. But shit went downhill
fast with his second, unfettered
reign when MAGA took total control
over the Republican Party.Now it
is 110% clear that the American
Dream is dead and buried, except
for the well connected wealthy,
and your military was state of the
art, two decades ago.The 21st
century belongs to China and
Europe.
|
> > > > > > mannanj
I don't know, I don't really buy
that this all went downhill with
Trump. It's easy to say that about
someone you don't like and fail to
overlook all the signs with
someone you like.
|
> > > > > > Gud
I didn't have a problem with Trump
during his first term.It was clear
he was unfit to be a president,
but he was mostly golfing.
|
> > > ltrg
> This vaguely-nationalist world view around
tech that's emerging in Europe is dangerous,
man.It's a response to the
actually-nationalist practice of the United
States. I can understand why it might feel
different from California, but things are a
bit scary over here right now.
|
> > > olmo23
> This vaguely-nationalist world view around
tech that's emerging in Europe is dangerous,
man.It's a direct response to the MAGA /
America First attitude of the American
electorate.
|
> > > blackoil
Collectively Europe has talent and money to
support it. Whole point of sovreignity around
it is to have inhouse the talent and capacity
to have essential resources even if it can be
have for cheap from other
nations/alliances.That is why people lament at
idea of being completely dependent on Russian
gas, US tech or Chinese manufacturing.
|
> > > trollbridge
The EU should absolutely be running frontier
labs producing frontier models.The fact they
aren't points to the moribund nature of things
in the EU tech space. Both China and America
are doing this although with very different
approaches.
|
> > > torginus
wdym? it says 13.5 million euros has been
allocated to the project - you aint building a
frontier LLM with that kind of money, but its
enough to give research opportunities and
real-world experience to a handful of
researchers, and build up local
competence.These people can then fund
for-profit companies once they have a
promising approach and bring in private
investment.As for nationalism, like it or not,
this is a govt sponsored effort, and
governments and universities are funded by the
public of specific nations
|
> > > > saidnooneever
this exactly. its not nationalistic at all
to support your local development. this
project has different goals than people
perceive and their comments are all
missing the plank ;)
|
> > > hrmon
> This vaguely-nationalist world view around
tech that's emerging in Europe is dangerous,
man.Guess which country blocked access to a
SOTA model based on national security
bullshit.
|
> > > legacynl
Hopefully you understand that there's an
obvious selection bias you encounter when
talking to folks from NL or DE in California.
Ofc the people that moved will think there's a
good reason for them moving. But people who
live here have good reasons for staying here.
|
> > > aoshifo
I don't see it as a nationalist world view
around tech in Europe. There is certainly a
push for "sovereignty", but that mostly means
sovereign from the US. But European countries
are actively working together on this.> Got
it, so every country should focus on having a
mediocre-at-best AI strategy by refusing to
work together? Surely this will create a
better future instead of pooling
resources.Yeah, why is Anthropic making their
own mediocre models, if they could just pool
their resources with OpenAI, that makes no
sense.
|
> > > notarobot123
> pooling resourcesYour argument would suggest
the EU developing a European model would be a
better direction. A heavy-weight competitor
would help advance the field after all.>
getting bureaucracy out of the way and
encouraging real investmentI don't think this
is really about regulation - it's about
network effects. The only way to compete with
strong network-effects is to create your own.>
This vaguely-nationalist world view around
techNationalism breeds nationalism and it is
the fundamental reason European states feel
the need to build their own expertise. Can you
imagine if your country was subject to the
whims of an aspiring dictator?
|
> > > woctordho
Users don't need to build wheels. Wheel
builders need to first learn to build some
mediocre wheels, then they can build better
ones.
|
> > > torginus
wdym? it says 13.5 million euros has been
allocated to the project - you aint building a
frontier LLM with that kind of money, but its
enough to give research opportunities and
real-world experience to a handful of
researchers, and build up local
competence.These people can then fund
for-profit companies once they have a
promising approach and bring in private
investment
|
> > > torginus
wdym? it says 13.5 million euros has been
allocated to the project - you aint building a
frontier LLM with that kind of money, but its
enough to give research opportunities and
real-world experience to a handful of
researchers, and build up local competence.
|
> > > throwaway98797
if something's strategically important for
safety it's wise to have competency in it.
|
> > > expedition32
Tech sovereignty is a response to American
imperialism. It has not gone unnoticed that
the White House has summoned the tech bros and
weaponised tech.
|
> > __alexs
Surely this just uses state funds to train a
pipeline of people to be immediately poached by
frontier labs?
|
> > yread
Great, now they will get expertise and then get
hired by openai and move
|
> mschuster91
Kimi and Qwen come out of China, which means that
their training material may be biased e.g. relating to
Taiwan [1]. In addition, there is no way to determine
what input went into the training, if it was properly
licensed, if it was legal (e.g. not contaminated by
CSAM), or how the human component of RLHF was sourced
- in US models, for example, stories about
exploitation like [2] have been floating for
years.Assuming us Europeans finally get our act
together, I think it is better for our long-term
future (and the ethical problems) if we manage to get
a baseline of training input and data ourselves, from
scratch, with everything being ethically sourced.Oh
and, while we're at it, the EU has 24 official
languages plus a host of minority languages. Most LLMs
focus on the English, German, French and Chinese
languages, but everything else is... left behind at
best. An European model with actual funding and proper
data sources might be able to significantly reduce
that.[1] https://www.taiwannews.com.tw/news/6245677[2]
https://www.theguardian.com/technology/2024/apr/16/tec
hscape...
|
> > altmanaltman
It really doesn't matter if the model sucks and
doesn't perform well. Given the funding amount and
their lofty ambitions, it seems very unlikely they
will be able to pull it off properly.Yeah China
and US models have baises but so will any model.
The biases do not get in the way of the product
though. You don't open those models just to ask
for what happened in Taianaman square or if Taiwan
is a state. You dont ask ChatGPT to generate CASM.
But they are very good at the tasks you actually
expect from a LLM. If you fail at that, nobody
will use your model no matter how "ethically
sourced" a colonizer-based entity like Europe made
it.
|
> > > edg5000
> no matter how "ethically sourced" a
colonizer-based entity like Europe made itThe
attempt is laughable, buy every country should
at least try to keep up with frontier
technology, even if they fail massively or are
massively underfunded.On the other hand, it's
arguably wasteful for an incompetent govt to
do something like this, since the money will
almost certainly not be well spent. It will
just go to people good with MS Word. That's
the likely failure mode for such NL innovation
projects. The actual solution is a culture
shift, but that is much harder if not
impossible to pull off and requires decades.
But we (NL people and govt) should work
towards that. Most likely all these govt led
innovation attempts are a sad waste of tax
money.
|
> > > > bigfudge
The culture shift that has generated this
is the same one that causes the other
story on HN this morning about xAIs gas
generators being a national security
issue. Ie one towards corruption graft and
the public ill.I don't want Europe to
model itself on the US, whatever the
economic gain. Hopefully we are large
enough to find a third way between China
and the US.
|
> > vintermann
The Chinese models are almost certainly taught to
comply with "Chinese values" in the RLHF step, not
from filtering the training data. There may be a
few things which are too radioactive to be allowed
even in the training material - but that's more
likely to be things like child abuse images for a
visual model, things non-Chinese values also have
an issue with.I'm pretty sure no county taking a
stab at making their own model for sovereignty
purposes will let "proper licensing" stand in
their way.
|
> > gnerd00
> Most LLMs focus on the English, German, French
and Chinese languages, but everything else is...
left behind at best.that is not true, so please
read before make an opinion. The French Mistral
project shipped seven+ years ago with 140
languages for example.. language translation was
the first LLM task from 2015
|
> > > selcuka
One example is not the same as "most LLMs". My
experience is the same with most LLMs.
Especially the smaller ones are English
oriented (probably makes sense given the size
constraints).
|
> > jampekka
> Most LLMs focus on the English, German, French
and Chinese languages, but everything else is...
left behind at bestCurrent frontier models (closed
and open) are already really good at small
languages too. I use them in Finnish sometimes,
and the language is immaculate. They underestand
even somewhat obscure dialects. Multilinguality
seems to be a mostly solved problem.
|
> > KronisLV
This already exists https://eurollm.io/How do
people not know about it and keep making stuff
from scratch?
|
> > > Alexander-Barth
I did not know about EuroLLM. I had a look to
the paper (https://arxiv.org/abs/2602.05879)
describing it:Specifically, we discard
documents shorter than 200 characters (Xue
et al., 2021a), and any page containing the
phrase "lorem ipsum," the word "javascript,"
or curly brackets (Raffel et al., 2023)....It
is quite surprising/funny to see all documents
with javascript removed.
|
> > dr_dshiv
There is something north of 8% OCR error rates..
that will hurt model quality!
|
> > siva7
Uh, some would say it's easy to determine what
input went into the training for kimi and qwen..
since they were caught stealing it from American
labs. Some cultural cliches may never change.
|
> > > ignoramous
> since they were caught stealing it from
American labs. Some cultural cliches may never
change.Has a formal lawsuit been brought to
bear? Given, Anthropic & OpenAI are being
dragged through courts for copyright violation
(or stealing, as you'd call it, if the
companies involved were culturally Chinese) by
newspapers, publishing houses etc; one'd think
they'd pass on some of that medicine to
Alibaba, which does have business entities
registered in the US.
|
> > > janc_
It's well-known that all commercial models are
based on stolen content. That doesn't mean
there is no filtering/censoring, just that the
censoring likely depends on where it's
happening...
|
> > > > selcuka
> It's well-known that all commercial
models are based on stolen content.Does
that mean that Chinese models are the
"Robin Hood"s of the AI era?
|
> > > kouteiheika
> since they were caught stealing it from
American labs...and "good guys" the American
labs were caught stealing from authors all
over the world[1].[1]:
www.npr.org/2025/09/05/g-s1-87367/anthropic-au
thors-settlement-pirated-chatbot-training-mate
rial
|
> > > > j_french
.... Anthropic began buying books in bulk,
tearing off the bindings and scanning each
page before feeding the digitized versions
into its AI model, according to court
documents.Wow. This image of Anthropic
employees ripping books apart to use them
to train models is a powerful one, seems
like an inflection point in the history of
information.
|
> > > basisword
>> Some cultural cliches may never
change.Let's just gloss over the monstrous
amount of copyrighted and pirated material the
American labs trained on. China bad. American
good. Some cultural cliches never change.
|
> > > > mschuster91
How about, both China and the US bad,
Europe at least somewhat decent because we
lack the financial incentives to behave
like utter arseholes?
|
> TJSomething
If open frontier models start closing up and states
start more export controls on AI services and
hardware, it might be good to ensure the supply chain
is there to reproduce the SotA, or even a couple
generations behind it.
|
> teekert
There is something to be said for this "most cheapest"
approach, there is also something to be said for
making models that are entirely ethically sourced:1.
Free of controversy like unlicensed training
materials2. Free of exploitative rlfh loops by people
in low-wages countries3. The leasons learned (and
published) from going through the entire training
process on "European" hardware: "AI factories" (the
term for Slurm HPC/HTC systems with lots of heavy GPU
nodes, heavily subsidized by our government [0])1 and
2 are strong counter-LLM arguments at the moment, and
hold back some groups of potential users. Another is
energy/water use, so going for maximum green energy
would be a nice boon as well. 3 is something I
consider to be highly useful for our European identity
and "way of the ninja" (for you Naruto fans out
there).[0 https://hpc-portal.eu/funding-opportunities]
|
> > dijksterhuis
one would also hope there'll be less pressure to
"make line go up", i.e. not having to do
attention-engineering via deliberate sycophancy to
trap individuals into using it more and more and
more and more and more and more.but in general,
yes, as someone who is vehemently anti-ai GPT-NL
has piqued my interest specifically because of the
ethical protections / measures they're talking
about. question is whether they stick to it.
|
> entropyneur
Sounds like calling those model "open source" did its
wicked job. You can not take an open weight model and
build a next-generation model using that as a
foundation. Once those companies decide it's no longer
in their interest to release new open weights
everything you've created this way becomes a pile of
rapidly deprecating legacy.
|
> > michaelscott
OP probably means using the existing open weights
as a base for further homegrown development and
research, not that the homegrown models are always
updated based on whatever US or China are doing in
the moment
|
> 627467
Do we know for sure how much national corpus of
knowledge (like dutch) goes into these "global" models
and how that affects "localized" model biases? What's
wrong with specialized models?
|
> saidnooneever
TNO doesn't serve nor represent global interest and
hence does not care about global progress. It exists
to enhance knowledge on things within Dutch society
primarily with some ripple effect outward to EU
because they have interests within the EU.Its purpose
is not to become some kind of OpenAI or global
foundation offering services/tools on that scale.There
is a lot of critisism on this project, not invalid,
but mostly based in lack of understanding of what the
goals are of the organization as well as the people
building the thing.The people building it, are well
aware of how it will be less capable than other LLMs
on a general reasoning aspect, not only due to having
actually purchased _all_ licensed data that has been
used as inputs. Not being a multi billion dollar
corporation, this means having very little data and
should be an obvious signal to observers that it has
not the goal to outdo other models.In my opinion
(personal) its a project that has a learning and
demonstration value that is not 'look how well our
model performs against others', but still offers
value.
|
> thevinter
And what happens once the "solid baselines" become
unavailable for a reason or the other?
|
> > zozbot234
You keep building on the last available version?
Fine tuning is a whole lot cheaper, easier and
more useful than pretraining a model from scratch.
It's a complete no brainer.
|
> > > rapidfl
> You keep building on the last available
version?yes but a sovereign can allocate some
resources and a few people to stay in the loop
from a first principles level. No need to wait
for a rug pull.Of course, it can not compete
with the frontier labs. But good to have
researchers and professors "in-house". LLMs
are here for the long-term.
|
> > > > GTP
> But good to have researchers and
professors "in-house".I'm not in this
field, but I think we already have them.
Probably the main difference is that we
have most or all of them in academia and
next to none insode private companies. But
we do have them, and they could start
working for private companies if the
market moves in that direction in the EU
as well
|
> > > > michaelscott
Unfortunately in this game first
principles requires massive resources, not
"some". Building in-house on top of
existing open weights is a good way to
bootstrap this process, especially since
there's nothing inherently magical or
particularly expertise-heavy when it comes
to weights themselves
|
> > ozim
Seems like you don't understand.You take current
version and build on top of it. You have the
weights.You might not get some n+1 version at some
point but the n version you will have will be
still most likely much better than whatever you
come up with burning good will money of people
believing in „sovereignty".You are not getting
ahead in this game by being „true to your local
values" capital expenditure is insane in this
game.
|
> > > oneshtein
It seems like you don't understand. For fine
tuning, it's cheaper to fine tune an existing
model. For massive changes, it's better to
retrain from scratch. Otherwise, model will
UNLEARN a lot first, and then you will train
about twice longer to the same
result.https://en.wikipedia.org/wiki/Catastrop
hic_interference
|
> khafra
Today, you keep seeing "sovereign" LMs that are
subject to the sovereignty of some human-led state.
Tomorrow, the "sovereign" LMs will be called that for
a completely different reason.
|
> Scarblac
Their legality is very questionable given all the
likely copyright infringement going on, and a state
can't really ignore that.
|
> > vintermann
States are the things which can ignore that, and
I'm pretty sure US and China already do. No state
is going to respect copyright if they think its
future is at stake, and apparently even
Netherlands thinks the future is at stake.(Of
course states can ignore copyright in a legally
polite manner, such as asserting that training on
all published material in the National Library is
fair game)
|
> enaaem
Same reasons why every country, or close allies, build
their own tanks or space program. You want to keep
some level of capability within your control. Compared
to weapon programs, AI research is very cheap.
|
sublimefire
It is crazy that anything Europe gets so much hate. IMO it
is important to build models within the boundaries of
smaller nations, using their own language. Research has to
continue even if it is outside of US and China.
|
> jampekka
I was somewhat excited about these "sovereign" open
models in the beginning, but it became soon apparent
that they're not gonna be anything but toys compared
to SOTA.The problem is that there are a lot, at least
30, of these small projects scattered around, funded
for a few years as some ad-hoc temporary coalition of
universities and businesses. Those simply cannot
compete with businesses spending tens of billions on
developing these. Especially when you have to bring a
spoon to a gunfight restricting to "clean"
data.Multilinguality is essentially a solved problem,
and restricting too much on one language with more
limited resources is gonna make the model worse in
that language too.
|
> > sublimefire
Anyone can move the needle. Saying that languages
are solved is not accurate as well. You could
raise different questions like maybe model
grounded in a different language will make it more
efficient in some tasks, maybe language structure
matters for a multidimentional space, maybe that
matters for the distillation, etc. It is all about
the ideas and their exchange, not about the
investment rounds and MAU.
|
> > Zababa
Even if multilinguality isn't solved, building a
benchmark and then testing each model on it and
posting the result may be a cheaper accelerator of
competence in the language.
|
> sinuhe69
This platform is running from the US and frequently
accessed by US people. What else would you expect?
|
> > andy12_
I'm from Spain and I also hate these projects with
passion. Creating models that speak multiple
languages is a solved problem. Having each
European Nation train its own useless "sovereign
model" in its own language is a total waste of
time and resources when we could pool resources
and give it a try to training SOTA models that
speak in all European languages.I'd rather have
smaller european labs try to give it a go at
distributed training. If multiple countries got
together and said, "look, we tried training a
distributed model that speaks in all of our local
languages and that is comparable to 1-year-old
Chinese open-source models", that, at least, I
would find interesting.
|
> > > Zambyte
Excuse my ignorance if by "distributed
training" you mean a specific process, but
couldn't this be considered a step toward
distributed training? If nations train models
independently and then later distill them into
a single model, all the work (both the compute
and the research processes) are distributed
for the initial training phase.
|
> > > > andy12_
I mean it as in, train a model across
different clusters instead of a
centralized cluster. It's been shown that
it's possible to train 10B models this
way. If more research effort was put into
this, that would be greatI don't think
your approach would work because you can't
create a strong model from distilling
several weak
models.https://www.primeintellect.ai/blog/
intellect-1https://www.primeintellect.ai/b
log/intellect-2-release
|
> mvc
self-interested hate is it not?Maybe I'm more attuned
to this type of thing having grown up as a national of
a smaller state living in the shadow of a bigger state
but you constantly see actors from the bigger state
belittling and condescending anything contributed
socially or economically from the smaller state.And I
see this sort of dynamic here in this forum where
Americans very frequently talk condescendingly like
this about Europe generally and European tech
especially (they did it to China too but China smartly
ignored this self-interested nonsense and carried on
anyway which is what Europeans should do).It really
grates on me and presumably many others. But it serves
an agenda too of a lot of the founders and financiers
that hang out here that have big fat customers in
Europe they'd like to keep sweet and competitors
they'd like to keep down.
|
> transcriptase
It's not that it gets hate so much as it's akin to
watching them make announcements that they're going to
make a European google/facebook/tiktok.Sure... they
can, except at the end of the day it's a bit late,
regulatory burden will make it comparatively useless,
and because of that nobody will ever use it. It will
be spending a bunch of taxpayer dollars for press
releases.The running joke is that when these
"sovereign" EU models launch, they're going to refuse
to answer anything that might involve personal
information such as Elon Musk's birthday.
|
> > arrrg
At least with social networks the network effect
is a powerful force. Foregrounding regulatory
burden in that context is nonsensical. (That does
not apply in the same way to models.)
|
> > data-ottawa
That's on Wikipedia, it's not PII, it's also not
going to be relevant to any meaningful IRL work.I
challenge the assumption you can do meaningful
work in this field without blatant disregard for
intellectual property.The idea that it's all down
to training size is clearly incorrect, as every
expert human learned their craft without nearly
the sum total information of the internet. Clearly
there are architectural wins to be found.Besides
that, why would everyone just be fine with Opus
level AI at best, as that's all the US is willing
to export, and I doubt China will share beyond
that.Sovereign AI is more important than ever
after Friday.
|
> > SiempreViernes
I guess if you are strict about it, making
derogatory comments like yours is indeed not hate.
But I'm sure you are aware that "getting hate"
frequently used in a more extensive meaning
online, especially in the context of replies to a
post and I don't see the much point in insisting
on the stricter definition here.
|
> > Lucasoato
I kinda agree, the best use of taxpayer money
should be in reducing taxes to corporation that
would like to compete in the market vs US and
China, rather than making governments playing the
game (since they very obviously can't).
|
> mholm
If a teenager on your street said he was going to
spend $1,000 to customize his Honda Civic for his
needs, you'd believe him. If he says he's going to
build a brand new car, better than a Honda civic, for
$10,000, you'd laugh and say good luck.
|
> > bigfudge
But if a teenager says he's going to spend 50k
going to university to study engineering you might
support then.I agree there is likely some hubris
in this sort of announcement, but investing in
European expertise and industrial base in this
area is important.
|
WarmWash
If Europe is serious about getting home grown AI fast,
three simple steps:1. Huge tax incentives, let the
companies get grossly wealthy while paying minimal taxes.
Minimum 10 years with clauses protecting "retribution"
taxes there after.2. Tax incentives for the
founders/shareholders, just like above.3. Drop worker
protections to a minimum, make it easy to fire people. You
only want serious/dedicated employees anyway.Within 2-3
years there will be at least a trillion dollars looking to
get in.Don't worry though if reading that made you mad.
Its absolutely not going to happen. I can think of few
things more antithetical to the European ethos than smart
skilled people working 80-100hrs weeks with almost no
vacation to gas their founders net worth by tens,
hundreds, of billions.
|
> dminik
Some additional points to consider:1. Pay the workers
in company scrip and relocate the workers to a company
town. That way, all workers are fully dedicated to the
company.2. Start importing slaves from Africa again.
It worked to build up massive wealth. Should do the
trick for AI as well.3. Abolish the 8 hour work day.
No comment needed.With these 3 simple tricks, you too
can get 6-7 bazillion euro AI mammoths.
|
> > WarmWash
No, we don't need any slavery. The employees at
these AI companies will almost certainly out earn
most if not all other local white collar workers,
while also getting top tier benefits. They can
also quit at any time if they don't like it or
don't think it is fair.
|
> > > dminik
We'll, someone has to do RLHF.
|
> TalkingCodeMonk
This "greed is good, and should be rewarded"
philosophy is one I see all too often on HN, and the
entire reason why Americas political, regulatory, and
business leadership have been overrun by the countries
most criminally corrupt narcissists and psychopaths;
why its democracy has collapsed.When you reward the
most selfish, corrupt, and antisocial behaviours with
wealth and power, you're guaranteed to create a
selfish, corrupt, and antisocial society. IMHO it's
indicative of what I have dubbed Americas "mental
illness epidemic"; specifically cluster B personality
disorders [0] which are characterised by
socially-destructive and self-destructive
behaviours.If that's the world you want for you and
your loved ones, congratulations. You've earned it![0]
https://wikipedia.org/wiki/Classification_of_personali
ty_dis...
|
> > user43928
If tax incentives to attract companies and at-will
employment are already viewed as a destructive
collapse of democracy, it's no wonder we are not
getting anything done here in Europe.
|
> > > TalkingCodeMonk
I suggest you take the time to read
independent analysis on the benefits of
corporate subsidies in America.I know I've
read of many cases where they cost cities and
communities far more than the economic
activity they promise to generate (eg Amazon
warehouses and sportsball stadiums).Subsidies
could make strategic sense in the case of AI,
but it's terrible economics to encourage
businesses that take more from communities
than they give back, especially those that
don't employ many people.
|
> > > > user43928
Data centers are known to create few local
jobs while requiring upgrades to the power
grid or water infrastructure.Wouldn't that
make it important to subsidize them at the
national level, precisely for that reason?
Otherwise it seems likely to be a NIMBY
problem.But now I'm talking about data
centers, the main thing we lack here is AI
research labs, is it not? I think those
should create many jobs.
|
> > > > > TalkingCodeMonk
Why should taxpayers anywhere pay
private enterprises to create
infrastructure they own and control
for-profit, rather than just cutting
out the monopoly middle man and
establishing that infrastructure as a
public utility who services the
domestic economy, equally?This is the
kind of madness I'm talking about. If
taxpayers have to subsidise products
and services the "free" market refuses
to provide, they should have direct
ownership in the profits; not just the
losses. Why is the default for society
to subsidise expenses and privatise
profits? That is ideologically-driven
dogmatic insanity, which allows select
individuals and foreign entities
control over the national economy and
its security, and is the entire reason
madmen in the USA have the EU (and
world) by the balls.
|
> > WarmWash
No greed will be rewarded. If European consumers
don't like the end AI models that are produced,
they can forgo buying them and the investors can
watch their fortress burn. For the EU taxpayer,
nothing was gained and nothing was lost. All the
state did was stand out of the way.If the models
are good though, they will have their sovereign AI
and should be happy to pay for it instead of
American or Chinese models. You may call it greed,
but to me it just sounds fair.
|
> > > TalkingCodeMonk
> For the EU taxpayer, nothing was gained and
nothing was lost.Except for the "massive"
subsidies you advocated for, right? Except for
the massive cost to provide utilities and
infrastructure to support those businesses?
Except for the cost to support the employees
they refuse to provide minimum "protections"
for? Except for any damage they do to the
local environment?You seem to think that
businesses exist in a vaccum where they have
no impact on society or its population, except
when they succeed and everyone should be
eternally grateful. This is the systemic greed
and narcissism America exports to the world;
not just through its businesses, but its media
and population across forums such as this one,
and is exactly what I'm talking about.
|
> > > > WarmWash
There would be no subsidies and the
companies would be responsible for funding
their own infrastructure. Protections for
workers would be minimal, because we
wouldn't really be interested in hiring
people who want strong protections. They
can stay at their current job. You balk at
this, but all of Europe's capable tech
minds left the EU to go work at tech in
the US over the last 30 years. They didn't
care about the generous protections, they
cared about making stuff and making money.
Which they did, overwhelmingly.What I am
describing is an existential battle for
the relevance of Europe on the world stage
in the future. Right now Europe is looking
like a vasal state, picking between the US
and China for who it's overlord will be.
If Europe wants independence, it needs to
mobilize immediately to catch up.
|
> > YetAnotherNick
3 Times As Many Europeans Move to the US, than the
Other Way Around.[1]:
https://mises.org/mises-wire/3-times-many-european
s-move-us-...
|