Packaging and the side-effects of it are always emotional amongst those of us who depend them.
That being said, I agree with you that that IPS has had its chance, and failed over and over again. It is frustrating, at-times cryptic and seems to require far more effort than managing the files myself would.
I appreciate that Illumos wishes to be package-neutral, but think that it might be time to (at the very least) mothball IPS and focus on the project's goals rather than fighting with a tool whose potential was far greater than its implementation.
And i'ld actually vote for dpkg. Its proven, well developed and the tooling around it is at least not bad.
And just like with Linux I actually do not see the value of a gazillion of packaging systems. Decide on one and go with it. Be it SVRr4, dpkg, rpm or bsd ports.
I'd say go back to SVR4 in the short term and look at migrating to DEB/RPM in the long-term. Both have supporting tools that are very mature and actively maintained. I don't see why reinventing the wheel would help here. It certainly hasn't for IPS.
Without specifics on what exactly went wrong, it's going to be hard to help.
I appreciate the concern, but expressing in terms of a bug report would be wonderful.
I don't imagine all the kinks are ironed out of either the ON build system or the way IPS interacts with it, but that said, the rest of the world doesn't build ON regularly: focusing on real-world usage of IPS obviously took priority I suspect.
Btw, if illumos were to consider an alternate format, it would probably be to revert to SVR4. The other formats have their plusses, but we already have the tools for SVR4, and ultimately, I don't want to introduce another chunk of probably -GPL'd code (for deb and rpm).
There's value in going back to SVR4 as well, since that's what most people using Linux have in production, and its what ISVs already are dealing with.
As you may know, IPS is under active development. If you construct package dependencies that cannot be resolved because they are mutually inconsistent, IPS doesn't do a very good job of telling the developer what went wrong.
Your statement about not being able to downgrade confuses me; IPS doesn't do downgrades. Instead, you should boot and older boot environment.
@Bart: yeah, the 4th year of development and still in development... If it is in development why is the observability and "debugability" of IPS so poor...
Good thing about IPS manifest is that it is very simple format and it is easy to convert it to something else. Few actions can be converted to shell scripts and the rest is mostly about deliverables.
As a package developer and maintainer for Solaris 10, all of my packages are in SVR4 and build by SS12 (about 60 pkgs), for me there are only two reason not to switch to IPS or what ever: Jumpstart and post/pre install/remove scripts.
Until there is a good replacement for Jumpstart I stick with SVR4.
It's unfortunate you encountered an issue, but unless you ask for help and tell us what specifically was wrong we can't help you.
At this stage in the project, most problems are a result of package issues, and not package system issues. The pkg(5) system isn't complete yet, so there are some rough areas. But I personally believe it has most of the core functionality one would expect.
You can always email pkg-discuss for assistance, or try on #pkg5 on freenode. And, as you already know, Rich Lowe is an expert in this area.
I *did* ask for help. Rich Lowe was working through the issues with me for several hours. And even the "expert" was struggling to make this work.
He may have a better idea on the nature of the root cause of the problems, but I'm not really sure.
Fundamentally, I can't risk illumos' future on a packaging system that has flag days that mean we can't package or install our own bits without hand-holding from a developer.
Yes, flag days can be disruptive, and an in-development project can be hard to track. But you already know this, so I find it difficult to reconcile why you believe this to be a stopper -- I'm truly puzzled by that.
Most community OS distributions that are based on an upstream project keep the native upstream's packaging system for a lot of good reasons.
I might also add that choosing a different packaging system than the one that will be used by Solaris will put IllumOS users at a serious disadvantage.
However, your target audience may be willing to put up with all of the shortcomings of SVR4 or the inability to confidently use native IPS packages meant for Solaris.
Binarycrusader: we'll still be using IPS metadata, and *distributions* that want to use IPS will be able to do so.
But I want to be able to build and the foundation code without dealing with breakage in pkg-gate.
I want users to be able to contribute without having to deal with the nightmarish scenarios I encountered.
The current idea is that nightly will be able to build packages in at least IPS, Debian, and SVR4. But only SVR4 will be enabled by default, and building the other two will require other components on the build system whereas the SVR4 bits will build using just the stuff in-tree. (And yeah, we can add targets for RPM, and random packaging format X too.)
But I want to be able to build and the foundation code without dealing with breakage in pkg-gate.
So pick a version of pkg(5) that works for your project and stick with it. It seems silly to argue against using IPS simply because it's in-development nature might inconvenience you.
So I'll assume that IllumOS is going to re-invent install, update, and zones mechanisms since all of those are now reliant on pkg(5).
As Tim and Shawn have indicated, grumbling about issues on blogs will only go so far. It's far more productive to file any sort of bug, even with scant details so that at least the issue can be tracked.
@Theo, I'd definitely appreciate hearing of the specific problems that you've had with IPS.
August 25, 2010 at 11:07 AM
Look, I really, really wanted to avoid entering the packaging debate. I mean, its an emotional decision, right? Well, its supposed to be. Except that I've spent nearly an entire day trying to figure out how to onu the latest illumos gate (which includes Rich Lowe's b147 merged in). I have gate changes that I desperately need to test in the context of a full install. (Well, I could say "screw it", and just test the bits in place -- which I've already done, but that's hardly a complete test.) I can't test them. Because I can't figure out how to use the packaging system to install them. And neither can our resident IPS expert, Rich Lowe. This is no longer an emotional decision for me. Yeah, there are a lot of "emotional" things not to like about IPS. (It forces a dependency upon Python; its still immature; it seems to fail if you are disconnected from the network; it doesn't seem possible to build and install "just" a single package; apparently there are a lot of magic incantations that nobody outside of the IPS developers really understands; etc.) I was willing to set aside all those "emotional" responses and use IPS, if it worked. If for no other reason than the fact that it did away with BFU I have been willing to give it my best effort. But the latest situation has left me dead in the water, and apparently NO ONE can help me. Look, I'm not a complete moron. (Well, maybe you disagree with me, but this is my blog.) I should be able to make this work. If I cannot, then what kind of barrier is this going to create for participation from other people? Is Rich Lowe going to hold the hands of everyone else to get past these issues? What happens the next time the pkg folks introduce another flag day? This is unacceptable. I'd like to hear other solutions. At the moment, I'm very very seriously considering gutting the IPS build requirements and having illumos go back to building SVR4 packages natively, using a tool to convert IPS meta data. (So meta data would be IPS, but binary deliverable would be auto-generated SVR4 packages.) The current situation reminds me of Linus' comments about CVS. I feel the same way about IPS right now. I'm very angry ... the tools that are supposed to facilitate development have caused it to cease for me. If the only way for me to move forward is to reinvent SVR4 build systems, then that's what I'll do. IPS is a failed science experiment. I don't see how it is going to get widespread adoption from anyone (ISVs or otherwise) with it as it stands today. Flames to /dev/null. Let me know if you have a solution though. Update: Rich was finally able to get me to the point of working. Although I can't ever downgrade. After what I just went through, I never want to. I'm really terrified that nobody really understands the steps it took to get me to a working state, and I am unwilling to force others to go through the same nightmare. So I'm still made at IPS, and I still think we need to unhitch the illumos cart from it.
posted by Garrett D'Amore at 10:02 AM on Aug 22, 2010
"IPS == FAIL"
20 Comments -
Did you consider using something like deb/apt?
August 22, 2010 at 11:04 AM
Packaging and the side-effects of it are always emotional amongst those of us who depend them.
That being said, I agree with you that that IPS has had its chance, and failed over and over again. It is frustrating, at-times cryptic and seems to require far more effort than managing the files myself would.
I appreciate that Illumos wishes to be package-neutral, but think that it might be time to (at the very least) mothball IPS and focus on the project's goals rather than fighting with a tool whose potential was far greater than its implementation.
+1 for progress over pedantics.
August 22, 2010 at 11:15 AM
I'ld second your opinion.
And i'ld actually vote for dpkg. Its proven, well developed and the tooling around it is at least not bad.
And just like with Linux I actually do not see the value of a gazillion of packaging systems. Decide on one and go with it.
Be it SVRr4, dpkg, rpm or bsd ports.
August 22, 2010 at 12:09 PM
Until there is a suitable replacement for Jumpstart, I'm all for SysV.
August 22, 2010 at 1:15 PM
something like pkgsrc ?
August 22, 2010 at 2:35 PM
I vote for anything that works. IPS has simply not worked well for us even after considerable investment.
August 22, 2010 at 2:40 PM
I'd say go back to SVR4 in the short term and look at migrating to DEB/RPM in the long-term. Both have supporting tools that are very mature and actively maintained. I don't see why reinventing the wheel would help here. It certainly hasn't for IPS.
August 22, 2010 at 4:37 PM
Without specifics on what exactly went wrong, it's going to be hard to help.
I appreciate the concern, but expressing in terms of a bug report would be wonderful.
I don't imagine all the kinks are ironed out of either the ON build system or the way IPS interacts with it, but that said, the rest of the world doesn't build ON regularly: focusing on real-world usage of IPS obviously took priority I suspect.
Seriously: bug reports welcome.
August 22, 2010 at 8:05 PM
Honestly, if I could figure out how to reproduce what went so horribly wrong, to construct a meaningful bug report, I'd do so.
I know that I had issues with onu, and Rich probably understands the problems better than I do.
A packaging system has to cater to both the people who install software, and to the people who *build* and package it.
August 22, 2010 at 9:37 PM
Btw, if illumos were to consider an alternate format, it would probably be to revert to SVR4. The other formats have their plusses, but we already have the tools for SVR4, and ultimately, I don't want to introduce another chunk of probably -GPL'd code (for deb and rpm).
There's value in going back to SVR4 as well, since that's what most people using Linux have in production, and its what ISVs already are dealing with.
August 22, 2010 at 9:39 PM
As you may know, IPS is under active development. If you construct package dependencies that cannot be resolved because they are mutually inconsistent, IPS doesn't do a very good job of telling the developer what went wrong.
Your statement about not being able to downgrade confuses me; IPS doesn't do downgrades. Instead, you should boot and older boot environment.
August 23, 2010 at 12:06 AM
@Bart: yeah, the 4th year of development and still in development... If it is in development why is the observability and "debugability" of IPS so poor...
Good thing about IPS manifest is that it is very simple format and it is easy to convert it to something else. Few actions can be converted to shell scripts and the rest is mostly about deliverables.
August 23, 2010 at 12:54 AM
As a package developer and maintainer for Solaris 10, all of my packages are in SVR4 and build by SS12 (about 60 pkgs), for me there are only two reason not to switch to IPS or what ever: Jumpstart and post/pre install/remove scripts.
Until there is a good replacement for Jumpstart I stick with SVR4.
August 23, 2010 at 2:45 AM
Garrett,
It's unfortunate you encountered an issue, but unless you ask for help and tell us what specifically was wrong we can't help you.
At this stage in the project, most problems are a result of package issues, and not package system issues. The pkg(5) system isn't complete yet, so there are some rough areas. But I personally believe it has most of the core functionality one would expect.
You can always email pkg-discuss for assistance, or try on #pkg5 on freenode. And, as you already know, Rich Lowe is an expert in this area.
August 23, 2010 at 11:07 AM
I *did* ask for help. Rich Lowe was working through the issues with me for several hours. And even the "expert" was struggling to make this work.
He may have a better idea on the nature of the root cause of the problems, but I'm not really sure.
Fundamentally, I can't risk illumos' future on a packaging system that has flag days that mean we can't package or install our own bits without hand-holding from a developer.
August 23, 2010 at 11:18 AM
Yes, flag days can be disruptive, and an in-development project can be hard to track. But you already know this, so I find it difficult to reconcile why you believe this to be a stopper -- I'm truly puzzled by that.
Most community OS distributions that are based on an upstream project keep the native upstream's packaging system for a lot of good reasons.
I might also add that choosing a different packaging system than the one that will be used by Solaris will put IllumOS users at a serious disadvantage.
However, your target audience may be willing to put up with all of the shortcomings of SVR4 or the inability to confidently use native IPS packages meant for Solaris.
August 23, 2010 at 11:46 AM
Binarycrusader: we'll still be using IPS metadata, and *distributions* that want to use IPS will be able to do so.
But I want to be able to build and the foundation code without dealing with breakage in pkg-gate.
I want users to be able to contribute without having to deal with the nightmarish scenarios I encountered.
The current idea is that nightly will be able to build packages in at least IPS, Debian, and SVR4. But only SVR4 will be enabled by default, and building the other two will require other components on the build system whereas the SVR4 bits will build using just the stuff in-tree. (And yeah, we can add targets for RPM, and random packaging format X too.)
August 23, 2010 at 12:32 PM
But I want to be able to build and the foundation code without dealing with breakage in pkg-gate.
So pick a version of pkg(5) that works for your project and stick with it. It seems silly to argue against using IPS simply because it's in-development nature might inconvenience you.
So I'll assume that IllumOS is going to re-invent install, update, and zones mechanisms since all of those are now reliant on pkg(5).
Have fun!
August 23, 2010 at 3:36 PM
Please see R1,Fix what SVR4 missed /outdated package management functions with wrapper by TWW toolset.
R1:https://www.illumos.org/projects/cpamtww
August 23, 2010 at 7:21 PM
As Tim and Shawn have indicated, grumbling about issues on blogs will only go so far. It's far more productive to file any sort of bug, even with scant details so that at least the issue can be tracked.
@Theo, I'd definitely appreciate hearing of the specific problems that you've had with IPS.
August 25, 2010 at 11:07 AM