Re: [Buildroot] [musl] Re: [musl] cortex-m support?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Buildroot] [musl] Re: [musl] cortex-m support?

Rob Landley
On 12/21/2016 12:18 AM, Waldemar Brodkorb wrote:
> Hi Rob,
> Rob Landley wrote,

Forwarding my reply to the old uClibc list instead of continuing on
buildroot and musl lists, since it's probably off topic there.

If anybody else is still listening, you can follow the previous messages
starting more or less at:

http://lists.busybox.net/pipermail/buildroot/2016-December/179782.html

>>> I am wondering why you don't cc me or any uclibc related list?
>>
>> I cc'd the buildroot list, which only has uClibc-based cortex-m support
>> at the moment. Why do you suppose I did that?
>
> That is not completely true, OpenADK has support for it, too.

Buildroot has cortex-m support based on OpenADK as well as its cortex-m
support based on uClibc?

> It seems to me that in your definition of a open source project,
> the people behind must either get money for their work or the
> project needs millions of installations.

I prefer to spend my free time on things that have a reasonable
probability of a future, yes.

When I first started banging on busybox and uClibc I was thinking "linux
from scratch is 110 megs based on gnu/crap but tomsrtbt does the same in
1.7 megs based on busybox, if I can save Knoppix 100 megs on their 700
meg CD they can work WONDERS with that". This was back when "Linux on
the Desktop" was still a feasible idea, and knoppix was leading that
charge. So there was a small but real chance it could help change the
world. (Plus getting Stallman's claws off the thing by producing a
version even he couldn't stick a GNU/ prefix on, at least without being
laughed off the stage, was a nice fringe benefit. My history with _that_
goes back to
http://web.archive.org/web/20060206061908/http://kerneltraffic.org/kernel-traffic/kt20020805_178.html#1
and I'm still sorry.)

Now I'm trying to turn Android into a self-hosting development
environment capable of rebuilding itself under itself, so when an 8 year
old girl in rural India inherits her mother's old castoff phone, buys an
old USB hub/mouse/keyboard at a yard sale with lunch money, and gets a
ten year old HDTV with a chromecast or something*, she can become a full
blown Android OS developer. (The economies of scale favor phones the way
they favored PCs 30 years ago, the old big iron technology will get rare
and expensive and the new thing will be cheap and ubiquitous, and this
will become more extreme as the new thing sucks away all the old thing's
users and use cases. Unless we want a read-only iPad future with no user
serviceable software parts, we need to lever open the phone the way
Compaq's clean-room clone levered open the IBM PC. There is _work_ to do
here to make it _better_.)

* USB to HDMI adapters require USB3, once that's ubiquitous they should
get cheap, but aren't yet.

Before either of those I spent half the 1990's trying to make OS/2
happen (because windows was terrible technology and basing the internet
on it meant not just a virus utopia but probably the kind of surveilance
state Edward Snowden eventually showed us we actually got.) My time on
OS/2 was a learning experience, and figuring out it was over and I
needed to move on to Linux in 1998 was an important lesson. I prefer to
spend my public development time on things that have at least the
possibility of a future. (I write plenty of code I don't bother to
_post_ anywhere. I don't need you for that, thanks. And $DAYJOB is a
black hole of unrelated programming time, for which I'm currently
supposed to be writing GPS signal processing software but it's the
holidays so I'm ignoring that and doing toybox for a bit instead.)

So yes, I was interested in uClibc when it had the distant but real
potential to change the world. But it sat there stagnating for TEN YEARS
during which the world changed out from under it, and these days its
world-changing potential is zero, and I have other things to do with
with my time.

Here's what happened in those ten years uClibc twiddled its thumbs,
starting with some research I did in 2003 helping defend IBM from the
SCO lawsuit:

  http://www.catb.org/esr/halloween/halloween9.html#id2867629

Which turned into a research paper about how Linux might take advantage
of the move from 32 to 64 bits to displace Windows on the desktop:

http://catb.org/esr/writings/world-domination/world-domination-201.html

Which didn't happen. I figured out I missed
http://landley.net/notes-2010.html#10-03-2010 in my analysis which gave
microsoft time to recover from Vista. On the Macintosh front (which was
at least a unix base) Steve Jobs' cancer made Apple back off from trying
to own the desktop instead of just cream-skimming the most profitable
20% (and of course he was ahead of everybody else with the iPhone). And
Linux on the Desktop continued not to happen because it turns out Open
Source development can't handle aesthetic issues the same way wikipedia
can't write a novel, as I explained at
http://landley.net/notes-2010.html#13-08-2010

By which time Smartphones were clearly doing to the PC what the PC did
to the Vax, and by 2010 I was strategizing how to survive the transition
to smartphones:

  http://landley.net/notes-2010.html#09-10-2010

Then in November 2011 Tim Bird made me reevaluate how the FSF's
licensing Jihad had opened up potential opportunities by making the GPL
so poisonous that previously unassailable existing deployed technology
was now being replaced:

  http://landley.net/notes-2011.html#13-11-2011

And I jumped in and started pushing. But if Android wasn't going to use
busybox because of GPL, it wasn't going to use uClibc because LGPL, and
as Rich was open sourcing his libc (because uClibc was dying) I talked
him into switching it to a BSD license while it was still all his code:

  http://landley.net/notes-2012.html#18-01-2012

Although incorporating it into my Aboriginal Linux project turned out to
be tricky because I'd frozen on the last GPLv2 toolchain versions and he
depended on stuff newer than that:

  http://landley.net/notes-2012.html#09-09-2012

But anyway, the last uClibc release I agitated for here was in 2012
because after that I was poking at musl, not uClibc. I was
_disappointed_ there wasn't another release, but not necessarily surprised.

As for money, you're the one who keeps bringing it up. Google's never
paid me a dime for toybox, I did busybox for free for years, and there
was only one year during the decade I spent pushing uClibc where it was
arguably job related and even then it wasn't what I was _supposed_ to be
working on.

Sigh. People have been mistaking me for a "group" for years:


http://www.zdnet.com/article/linux-group-calls-for-appointment-of-penguin-patch-lieutenant/

They were SO SURE that toybox was a Sony plot:

  https://lwn.net/Articles/478308/

Sony never gave me a dime either. (They flew me to ELC a couple times
back when it was called CELF, and paid for my hotel. But that was as a
speaker, and I paid my own way there after the first couple times. it
predated Tim noticing Toybox.)

Oh, Google did give everybody in the keynote audience at ELC 2010 a
Nexus One, and I was in that audience. And a few years later, said
keynote speaker (Google's head of Open Source) assured me that Google
had no interest in toybox:

  https://twitter.com/cdibona/status/342314738660810752

The difference between me and a normal person is I didn't stop work on
toybox when he said that. As Woody Allen said, 80% of life is showing
up. Maybe similarly persisting with uClibc will work out for you, I just
don't see how. (I have no idea what you're trying to accomplish. What
niche it has these days that musl, bionic, and (e)glibc _don't_ have.)

You're accusing me of choosing not to spend my free time on uClibc
anymore. I don't understand why this is some sort of accusation. There
are 7 billion people on this planet ignoring you. My being among them is
not an attack on your project. I pushed uClibc for 10 years (again, as a
hobbyist). I stopped when Rich posted his libc archive somewhere public
I could test and contribute to, which offered a better use of my time.

>> Did you want me to send it to the uclibc.org mailing list which hasn't
>> had a single post this month except your announcement of your fork's
>> release?
>
> No, I would like to see your suggestions and patches on
> the uClibc-ng mailing list:
> http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel/

As I told Thomas, I hadn't noticed the _existence_ of that list until he
pointed it out to me. I'm still subscribed to the old uclibc list for
the same historical reasons I'm still subscribed to the User Mode Linux,
tinycc, and gentoo-embedded mailing lists. I glance in those folders
maybe once a month. (Same for busybox but that one I occasionaly reply
to because it's sort of my alma mater even though I graduated. :)

There are _interesting_ revivals of old technology. I'm quite interested
in https://pdos.csail.mit.edu/6.828/2016/xv6.html for example, which
boils down to "a generation of students learned from the Lions book
which described Unix v6 for the PDP-11, but the code's K&R C and nobody
has a PDP-11 anymore, so let's update it to ANSI C on a PC". I'd love to
port that to to the j-core.org stuff I do at $DAYJOB because I think the
two projects would work well together to teach hardware _and_ OS design.
But I haven't had time to poke at it in the past year.

Good luck with your thing. The computer historian in me would happily
discuss it at length, but the programmer is not interested.

>> Has your fork solved the nptl issue?
>>
>>   http://lists.busybox.net/pipermail/uclibc/2008-September/020151.html
>>   http://lists.busybox.net/pipermail/uclibc/2008-September/020169.html
>>   http://lists.busybox.net/pipermail/uclibc/2008-September/020171.html
>>   http://lists.busybox.net/pipermail/uclibc/2008-October/041201.html
>
> Not sure. But in uClibc-ng there is Linuxthreads support for every
> supported architecture (excluding METAG) and NPTL for the global
> players. I added NPTL support from GNU libc recently,

That would be the pthreads snapshot from glibc 2.1.3 circa 2002. (uClibc
git commit e356ea321c80.) Renamed to linuxthreads.old in 2005 (commit
a9f5aa1cc96f) to make way for a second snapshot that was abandoned and
yanked _back_ out it didn't work.

I literally had this fight eight years ago:

http://lists.busybox.net/pipermail/uclibc/2008-September/041052.html

And yes, Bernhard's big goal for the 0.9.31 release was finally dealing
with the big pending treewide threading cleanup and unification:

http://lists.busybox.net/pipermail/uclibc/2008-September/041055.html

The fact you haven't managed it either says to me that your changes are
a coat of paint over dry rot.

> but Microblaze
> seems indeed bitrotted inside GNU libc as nobody is doing any test
> suite runs.
> https://sourceware.org/ml/libc-alpha/2016-07/msg00640.html

You're aware that half the point of my
http://landley.net/aboriginal/about.html project was regression testing
each new release under qemu to see what broke, right?

My main regression test being an automated Linux From Scratch build, the
same way the kernel developers used to build kernels under each new
kernel to see if anything broke. My setup had distcc calling out to the
cross compiler on the host through the virtual network, emulated block
device with ext3fs on it, and so on. And yes, I sent fixes to the
kernel, to uClibc, to qemu, collected local gcc patches, wound up
MAINTAINING busybox as a side effect of that project... pretty much
every release broke _something_.

Been there, done that. Doing other things now. The point is I am aware
how much work proper regression testing is. I also believe there's no
substitute for actual real-world use, and users other than the
developers who will find stuff the developers never will. The cortex-m
bug says to me that uClibc, on cortex-m, hasn't actually got serious
users or you'd have noticed the problem yourself years ago.

Maybe that will change, who knows? Meanwhile, I'm paying attention to
musl and bionic.

>> Do you have a sane "make defconfig" that lets people build uClibc
>> without learning what over a hundred individual config options do and
>> making decisions about whether or not they need each one? This issue
>> doesn't even come _up_ with musl, it fundamentally avoids most of the
>> structural problems that strangled uClibc development, by design.
>
> I regulary cleanup needless options and keep good defaults.
> The number of options get lower. I still like the idea of being able
> to build a complete toolchain without thread support or other
> stuff disabled. I will try to remove more options in the future.

There might be a ./configure --disable-threads in musl, but I haven't
bothered to play with it if so? (I haven't agitated for one.)

>>> You still believe uClibc projects should die?
>>
>> No, I believe uClibc _already_ died. I believe this because I was there.
>
> Yes, uClibc is dead. uClibc-ng is alive!

One's pining for the fijords, the other thinks it's going for a walk and
feels happy. Got it.

Have fun?

>> But cortex-m still only supports pthreads in 2016 and even that's buggy
>> in ways that were fixed out of tree quite a while ago. The release I
>> fished this bugfix out of is a year old. I don't have their source
>> control to see how old the fix really is, but emcraft's "preferred"
>> kernel (https://github.com/emcraftsystems/linux-emcraft) forked off from
>> mainline 7 years ago, and cortex-m support for Linux is their core
>> business, so there's a guess how long ago somebody actually _using_ this
>> might have noticed it.
>
> So, may be you can prepare a patch for the uClibc-ng list?
> Or do you want to see the bug open for another year?

I don't care? I did that for ten years, then I moved on.

I'm not going back to OS/2 either. And yes, such a thing would
technically be possible:

https://tech.slashdot.org/story/15/11/03/1337254/the-return-of-os2-warp-set-for-2016

Heck, I tried to convince Timur Tabi to take another look at Linux after
meeting him in the Armadillocon game room ~15 years ago; he told me he
liked being a big fish in the OS/2 small pond at the time, a couple
years later he was maintaining a largeish chunk of the Linux PPC
infrastructure (something like their PCI bus implementation?). Clearly I
encouraged him in that direction because I hated OS/2, not because I
thought there was a better use of his skills.

>> In case you really _don't_ know the history, let me walk you through how
>> the uClibc project died, going back through about ten years of
>> accumulated scar tissue, and why three different maintainers before you
>> failed to fix it.
>
> Thanks for the history walkthrough. Do you know why Bernhard is so
> unresponsive and more about the time shortly after his last release?

Do you mean you asked him questions he didn't respond to? I didn't ask
him questions, I just left. But if you're wondering why there wasn't a
uClibc release after May 2012, guess when I stopped agitating for one?

Look: Rich Felker used to hang out on the busybox list back when I was
maintainer, here he was in 2006:

  http://lists.busybox.net/pipermail/busybox/2006-August/057612.html

He had all sorts of projects he wrote but never published. Over the
years I occasionally asked to see some of them, and they were good but
pretty standard "lone wolf in a corner" projects (one developer, no
docs). One of these projects was most of a C library, about the same way
I'd done the first half of toybox but busybox was filling that niche so
I didn't push it much back then.

Rich and I hung out on IRC on and off for years (back when #uclibc was
busybox, uclibc, and buildroot together), and we would occasionally talk
about the state of uClibc, and I'd go "You wrote your own C library from
scratch, ever going to publish it?". But he'd also written his own xterm
(with much better utf8 handling support) and so on. It didn't really
stand out as a thing at the time. Then around when I was thinking about
reviving toybox in 2011, Rich finally decided to take the plunge and put
his libc into proper public source control and put it online.

I don't think I particularly talked him into posting it (he was as
frustrated about the perpetual zombie state of uClibc as I was), but I
do remember he asked me a lot of questions about being a maintainer,
although he quickly became better at it than I ever was. (Waaaaay more
patience and diplomacy than I've ever managed.) When he agreed with my
reasoning about BSD licensing being better for musl than lgpl, that
meant musl could potentially replace bionic in Android, and uClibc just
stopped _mattering_ to me.

*Shrug* uClibc _could_ have had a release since then, but they didn't.
Meanwhile, musl went from "git init" to a 1.0 release in about 3 years.
The guy who did it is still around maintaining it, answering questions,
and making fresh architectural decisions. There is no part of the
codebase that is not fully understood by its current developers, who can
change anything they like whenever they like without fear of breaking
stuff, because they understand all the ramifications.

Will that EVER be the case for uClibc? Do you know what a path to that
would even involve?

>> You came into a project I'd pushed for 10 years and started collecting
>> trivial bugfixes without addressing any of the serious architectural
>> issues (like needlessly duplicated per-architecture headers snapshot
>> from ancient glibc versions, with a #define pretending to be glibc (musl
>> doesn't) and declaring all sorts of stuff the library doesn't actually
>> implement). And you're wondering why I have a more pessimistic outlook
>> of your chances than you do?
>
> You have to start somewhere, otherwise nothing happens.

http://www.galactanet.com/comic/view.php?strip=312

(Yes, same Andy Weir who wrote "The Martian". After that webcomic he did
http://www.cheshirecrossing.net/ and then did prose on livejournal for a
bit before tackling an actual novel. It seems to have worked out for him.)

>> The big elephant in the development room was the NPTL code, and cortex-m
>> is still using pthreads, not nptl, and WITHIN that pthreads mess is _old
>> and _new versions of thread wait for supporting 2.2 kernels. (The
>> menuconfig option has been replaced with _old and _new suffixes on the
>> functions. Great.)
>
> Linuxthreads.new is gone. uClibc-ng only support Linuxthreads and
> NPTL.

So yes, a snapshot of glibc from 2002, and your upgrade is to copy NPTL
_from_glibc_, but only into some architectures.

The musl implementation is all new, supporting NPTL (I.E. futexes and
TLS) on all architectures. Which of the two is more interesting?

>> You've said that your reason for supporting uclibc-ng was for two
>> architectures, ARC and Xtensa:
>>
>>   http://www.openwall.com/lists/musl/2015/05/30/1
>  
>> That is why _you_ care. I personally think that adding support for those
>> to musl-libc would be a far better use of your time, but it's not my job
>> to tell you want to do. It's your hobby time. Do what you like with it.
>> Just don't expect me to take you seriously after ten years of this.
>
> That is not entirely right. I care for every supported uClibc-ng
> architecture.

I care about global warming. I was born in Florida and grew up on
Kwajalein and can reasonably expect to live to see both of them
disappear underwater. Miami's already flooding regularly:

http://www.miamiherald.com/news/local/environment/article107959967.html

(You can't put a sea wall around it, the entire state is on porous
limestone. When the tide's high enough salt water bubbles up out of the
ground half a mile inland in low-lying areas.)

But "I care" and "I can personally do something about it" are two
different things.

> uClibc-ng might be used for different use cases:
> - old non mainstream architectures like lm32, avr32, cris, fr-v, ..
> - noMMU systems like arm, coldfire, bfin, c6x, ..

Rich and I worked together to get nommu SH2 support into musl, and we're
working on cortex-m now. Once you've got the first nommu target in, the
rest are mostly a question of setting up the right test environments.

(Although Rich went with elf-based fdpic instead of a.out-based binflt,
which means for cortex-m on musl we've got to get the
http://www.slideshare.net/linaroorg/sfo15406-arm-fdpic-toolset-kernel-libraries-for-cortexm-cortexr-mmuless-cores
stuff upstream.)

> - architectures not supported by any other C library like arc,
>   xtensa, nds32, h8300, ..
> - vintage unix hardware like SGI O2, Sparcstations, ...
>
> See my table which architectures are not supported by musl/glibc:
> http://uclibc-ng.org/wiki/matrix

Good to know.

I've had a todo item to get a coldfire target running under qemu so Rich
can do musl support in it. It's been on my todo list for... 2 years now?
Too much else to do...

> Indeed Max has started a musl implementaton for Xtensa:
> https://github.com/jcmvbkbc/musl-xtensa
>
> The Synopsys people trying to add support for GNU libc.
>
> But even then uClibc-ng is always a good choice for regression
> testing and testing between the different C libraries.

Good luck with it.

>> I cc'd buildroot. That is a real project, which still uses uClibc for
>> the targets Rich hasn't gotten around to porting musl to yet. I cc'd
>> them so they would be aware of the issue. I could have just sent it to
>> the musl list, but chose to inform buildroot as well.
>
> Other real projects use uClibc-ng, too. So may be next time you sent
> it to our list directly ;)

I am not interested in subscribing to your list just now, thanks.

> best regards
>  Waldemar

Rob
_______________________________________________
uClibc mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/uclibc
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Buildroot] [musl] Re: [musl] cortex-m support?

Waldemar Brodkorb-4
Hi Rob,
Rob Landley wrote,

> On 12/21/2016 12:18 AM, Waldemar Brodkorb wrote:
> > Hi Rob,
> > Rob Landley wrote,
>
> >>> I am wondering why you don't cc me or any uclibc related list?
> >>
> >> I cc'd the buildroot list, which only has uClibc-based cortex-m support
> >> at the moment. Why do you suppose I did that?
> >
> > That is not completely true, OpenADK has support for it, too.
>
> Buildroot has cortex-m support based on OpenADK as well as its cortex-m
> support based on uClibc?

Yes, that is true. We found some problems with Linuxthreads and
could fix it upstream in elf2flt. Buildroot has furthermore good
support for STM32F29 devices. The Qemu-ARM noMMU default config was
contributed by me. OpenADK tries to support Kinetis K70 from
Emcraft, but it seems my compiler is to recent to compile their old
2.6.33 kernel correctly.
 
> Now I'm trying to turn Android into a self-hosting development
> environment ...

I dislike the hardware and closed source drivers. It is sad that
OpenMoko didn't evolved. People like me working on old crappy C
libraries, supporting crappy architectures and devices still own
such phones.
 
> Before either of those I spent half the 1990's trying to make OS/2
> happen (because windows was terrible technology and basing the internet
> on it meant not just a virus utopia but probably the kind of surveilance
> state Edward Snowden eventually showed us we actually got.) My time on
> OS/2 was a learning experience, and figuring out it was over and I
> needed to move on to Linux in 1998 was an important lesson. I prefer to
> spend my public development time on things that have at least the
> possibility of a future.

I started playing with Linux in nearly the same year. I think SuSE
Linux 5.2 was my first playground.

> Maybe similarly persisting with uClibc will work out for you, I just
> don't see how. (I have no idea what you're trying to accomplish. What
> niche it has these days that musl, bionic, and (e)glibc _don't_ have.)

You will see, probably.
 
> You're accusing me of choosing not to spend my free time on uClibc
> anymore. I don't understand why this is some sort of accusation. There
> are 7 billion people on this planet ignoring you. My being among them is
> not an attack on your project. I pushed uClibc for 10 years (again, as a
> hobbyist). I stopped when Rich posted his libc archive somewhere public
> I could test and contribute to, which offered a better use of my time.

Yeah, but the 7 billion people don't blog or post bad propaganda
about uClibc-ng. But even bad talk about uClibc makes it alive ;)

> You're aware that half the point of my
> http://landley.net/aboriginal/about.html project was regression testing
> each new release under qemu to see what broke, right?

Yes, I am aware of it.
 
> Been there, done that. Doing other things now. The point is I am aware
> how much work proper regression testing is. I also believe there's no
> substitute for actual real-world use, and users other than the
> developers who will find stuff the developers never will. The cortex-m
> bug says to me that uClibc, on cortex-m, hasn't actually got serious
> users or you'd have noticed the problem yourself years ago.

I hope to get the Emcraft developers involved in uClibc-ng
development and make things better in the future. We will see.
 
> > Thanks for the history walkthrough. Do you know why Bernhard is so
> > unresponsive and more about the time shortly after his last release?
>
> Do you mean you asked him questions he didn't respond to?

Yes. I even got a respond from Erik in the same mail, saying that
Bernhard is the maintainer.

> (Although Rich went with elf-based fdpic instead of a.out-based binflt,
> which means for cortex-m on musl we've got to get the
> http://www.slideshare.net/linaroorg/sfo15406-arm-fdpic-toolset-kernel-libraries-for-cortexm-cortexr-mmuless-cores
> stuff upstream.)

I am looking forward to see this and as usual I will test new ports
of musl. http://tests.embedded-test.org/musl/
 

> > - architectures not supported by any other C library like arc,
> >   xtensa, nds32, h8300, ..
> > - vintage unix hardware like SGI O2, Sparcstations, ...
> >
> > See my table which architectures are not supported by musl/glibc:
> > http://uclibc-ng.org/wiki/matrix
>
> Good to know.
>
> I've had a todo item to get a coldfire target running under qemu so Rich
> can do musl support in it. It's been on my todo list for... 2 years now?
> Too much else to do...

You can just use buildroot to generate a working environment, I
contributed a defconfig for it.
 
> > But even then uClibc-ng is always a good choice for regression
> > testing and testing between the different C libraries.
>
> Good luck with it.

Thanks.
 
> I am not interested in subscribing to your list just now, thanks.

That is fine for me.
 Waldemar
_______________________________________________
uClibc mailing list
[hidden email]
http://lists.busybox.net/mailman/listinfo/uclibc
Loading...