OSNews

The test is still relevant, as most users will not recompile their software for better optimizations. They will just use what they get from apt-get.

the worst part is, most people believe 100% what random bozos put on their blogs, as it if was absolute truth.

And are you claiming then that the blogger's result were false? He was receiving lower performance in several important applications under Linux than under XP. I do suspect it's the CFS scheduler which just tries to balance everything instead of giving one application way much more CPU time than another one. Linux did beat Windows hands-down in multitasking in his test after all.

I think I'll do a similar test on my laptop, though I'm gonna test Gentoo, Mandriva and XP. If Linux scores lower then are you going to claim my results false, too?

IMHO his tests were pretty much OK and valid. It doesn't really matter what is the cause in these things, it's the end-result which matters. One can debate things for all eternity if one wishes but no end-user will care about that.

Of course it matters - we can't begin to bring Ubuntu up to speed until we know the cause

I'll be interested in seeing your results when/if you perform these tests

I'll be interested in seeing your results when/if you perform these tests

It takes me a while to install anything on the laptop, it's pretty slow afterall, so it might take a few days. And I don't yet know where I'll put the results since I don't have a blog.

You could always consider writing a small article for OSNews with the results. I'm sure Thom would appreciate it, not to mention quite a few fellow readers.

when did i ever claim his results false? what i claimed was false INFORMATION, relating to why something might be slower/faster, and what causes instability.

i think the test results were reasonably fair. Taking a fairly unmodified out of the box Ubuntu install is reasonable. I think if he compiled any of the applications or the kernel it would have been cheating seeing that those who are not so tech-savvy would not even consider any extra configuration. I think tests like these are useful to show that much need to be done on the default settings of distros to make sure they run well for the most popular applications. Good test and I look forward to seeing the results of other distros. Thx

Yup, you're right, I mixed them up. It's fixed in the item now.

Thanks!

I don't know. I have ubuntu 8.04 running as a virtual host and its doing a really good job so far. Performance is good and I've had no reliability problems.

I fortunately don't need 3D so i'm quite happy with Fedora 9 and KDE 4.* .

I rather game on my xbox360 anyway :-)

I'm really not sure what you're talking about. A long-time Gentoo user who desperately needed a non-source-based distro for my new Macbook Pro, with much reluctance I gave Kubuntu-kde4-8.04 a try, and have been (decently) pleased.

Is there some story, some blog post, some link that you can give us telling us why 8.04 is the clusterf@%k you claim it to be?

I'd be interested to hear any intelligent criticism, much more so than unsubstantiated claims.

Not sure what's causing you problems - 8.04 seems to work well for me. Perhaps you have some "interesting" hardware?

But one of the features of free software is that you can stay with 7.10 for as long as you like if that works better for you, unlike certain other environments I know (*cough* XP *cough*).

XP - supported since 2001. Before it is done, it will wind up having being supported for 7-8 years. On the other hand, Ubuntu 7.10 will only be supported for a total of 1.5 years. I hope your cough get better.

You missed my point rather badly.

I can still download and install the original Ubuntu 4.10 today (get it at http://old-releases.ubuntu.com/releases/4.10/), while XP will die on 30 June (except for a few markets where Vista is hopelessly unable to compete) simply because Microsoft wants it to die. And Microsoft can deactivate *your* permission to run XP without recourse, as they did my daughter. Read your EULA.

And lauding Microsoft for supporting XP for 7 years is disingenuous and a bit laughable - it was Microsoft's ONLY OS for 5 long, delay-filled years until Vista shipped to a less-than-glowing reception. Of COURSE they supported it for what seemed like eternity!

Canonical ships new Ubuntu releases every 6 months with all the reliability of a Japanese passenger train, so they are supporting each release for 2 full successor lifecycles (*6* for Long Term Support releases like 8.04).

Hope that clarifies why Microsoft makes me cough. :-)

If i'm working with Blender I don't give a *whatever* CFS is doing - I need to finish my work in time. So Linux sux. If Ubuntu is positioned as Desktop OS it should work like desktop OS. If i will need server OS for heavy miltitasking i will use one that suits me right, but when i need desktop os to work with multimedia linux is no good for me.

then up the priority on blender...

This looks like a job for Renice -20 Man!


Faster than a runaway process!
Able to overcome scroogy schedulers with a single command!

Renice -20 man! Renice -20 man! he renice -20 you! He renice -20 me! Renice -20 man!

funny thing is that even with a -20 process, it will not give the same level of "lockup" as a realtime scheduled windows process. but thats highly iirc...

You don't get the gist of the compromises being made here. Windows multi-tasks and does some renicing in favour of the active application currently running, automatically. The problem with this is that the foreground running program, especially one as intensive as Blender, tends to drag down the rest of the system as a result. As soon as you multi-task to something else, poof, everything runs slower. Will that help you get your work done faster? Thankfully, this doesn't happen on any Linux system I've yet used.

However, in this case I doubt whether a renice would help Blender. There are a few other performance bottlenecks to look at first before doing that.

I don't buy your reasoning. If a scheduler degrades performance when you only run one application, then the scheduler is broken. I hardly think that is true for modern Linux schedulers.

Then you just claim that Linux is similar to the Vista kernel but better without any reasoning or proof.

"I like Linux more than Windows therefore it's better".

but you never run just one application, or do you?
all services and such are also applications. i guess it would be a bit hard for the scheduler to know exactly wich application the user preffers. well thats what nice is for. but then ordinary users dont want to fiddle with such things. maby desktop distributions should make so all desktop apps get higer priority. or add so users could chose priority by rigth klicking on a window

http://kerneltrap.org/Linux/Defining_Scheduler_Task_Groups

i know there is a better write up about it out there somewhere, but right now i cant find it...

***YAWN***

How many times have we heard this?

Some people are sick of intransigence, if you want to be tied to a certain application, fair enough, but do not expect people to stop the world to pander to you and your needs.

What I like about this is how you said all this was irrelevant to you above, but then complain about people complaining about how the world should pander to them.

Classy and consistent.

You picked me up worng there...

The first post was purely because the benchmarks are irrelevant to me, and in all honestly, to everyone else to. Benchmarks are never a true life reflection.

My second post was just a reply back to the usual posting that "linux is crap because Photoshop/Dreamweaver/ do not install natively".... same old story, same old tune. Some people expect the world to pander to their needs.

I may have read you wrong. And I certainly don't disagree with the "pander to me" attitude; Ubuntu 8.04 has been taking some hard knocks on the forums for the same reason.

In a way, I agree with this. It doesn't matter what operating system you use or how slow it is, as long as you can do your work.

In general, Linux boxes run lighter and faster and if someone is prepared to alter their work habits to work with Linux, they can get a very nice experience.

I can't imagine anyone thinking that they can get more performance out of Windows, whatever the release, than with Linux. All the optimisations in application software and Windows' bias to give more priority to the foreground application help, but when I quit an application and it takes several seconds for the icons on the desktop to be regenerated, I have to laugh.

I still believe that there are a lot of credible blogs and one only need to have faith in people.

Just to defend the guy, I'd say we have the right places where faith matters. In such cases credibility is what matters. And, however deep they say we are in the web2 mud, I have yet to get convinced by the credibility of the random blogger. In any issue whatsoever. That doesn't mean there aren't incredibly good ones, but you'd bettet take everything with a - grand - grain of salt.

Finally, you can do a similar test yourself, it is very simple. I can assure you that you'll get similar results.

I'd say not, I'm pretty sure one could produce fairly different results with a recompiled kernel and recompiled apps, or on a well compiled Gentoo, whichever.

Edited 2008-05-21 11:32 UTC

I think biggest problem is you cannot consider such data as absolute. Custom-configured machines are heavily dependent on personal preferences (for ex. did you buy specific hardware because you needed it to work with Linux? Maybe Windows with other hardware is blazing faster... maybe not... but this is an example of what I meant), constrains and so on.

Moreover, you cannot assume same application is optimized for different platforms the same way. By using the same application on both platforms, you cannot assume both versions are optimized and working at maximum speed they could do. Think QT and GTK: would you state that those ARE optimized for Windows platform? I don't think you can state that and that would modify your overall results.

I'm afraid that those data only demonstrate that, for applications you're using on hardware you own and for usage you're willing to do by them, Ubuntu MIGHT be faster. And we're happy for you.

I'm sure I can find thousands of combination where Windows can be faster. Plus, remember that many important details cannot be reproduced with a few tests but require more analytic approach.

Overall, your info can be useful to people willing to work with your hardware and willing to use same software you use. But you cannot use them to measure actual OS performance.

Typically, anything dealing with lots of small files will crawl on Windows, where the overhead of actually opening a file is very high. Similar problems occur with large directories (more than 1000 files or so). Either could have happened with the ClamAV test, and that might begin to explain the discrepancy.

Blaming NTFS itself might be a bit hasty, but there's definitely something going on in the Windows filesystem somewhere. The same issues do not seem to effect Linux nearly as much.

I wonder what was going on with the Blender benchmark though. Much of Blender's performance is dependent on OpenGL performance, so was the problem to do with nVidia's drivers, or the desktop effects which are present on Ubuntu but not Windows XP.

I cannot vouch for the CalmAV benchmark authenticity - but I can offer numbers of my own.
A certain I/O benchmarking tool (C, POSIX/Win32) that I've developed was designed to simulate high I/O using very high number of small files (1K-20K) running concurrent create/write/close cycles under different directories.

On the same machine (2x2C-Xeon, 4GB, SCSI RAID5) - under Windows 2K3 (i386/NTFS/no-8.3-names), the application was hard pressed to pass the 5MB/s line. Under Linux (CentOS5, ext3 [noatime/nodiratime], SELinux enabled) the application managed to sustain around 60MB/s.
Switching to FAT32 did improve the performance (~12MB/s) but I doubt that using FAT32 is a desirable solution.

After doing some additional research it seemed that:
A. CreateFile is dead slow. Around 10-100 times (!!!) slower then open/Linux.
B. NTFS performs OK as -long- as you keep the file count (per directory) to <100.
C. Once you pass the 100 line things get far worse, -fast-.
D. NTFS goes into what-looks-like a winter hibernation once you pass the 10,000 file/directory line.
E. NTFS doesn't seem to like deep directory hierarchy (If you rather replace file count with directory count). The deeper you go, the slower things get.

You can argue that I'm a poor Win32 developer or that I've missed anything (beyond disabling 8.3 filenames) - but you can try it yourself.

- Gilboa

Could you send me the benchmark pls? I'd like to try it on 2008 and post results.

Send to dikrek@gmail.com pls.

Thx a lot

D

I can only send you pseudo code - the code itself belongs to my employer.

- Gilboa

Well the one doing the benchmark could absolutely optimize both systems. But that wouldn't make sense for non-technical readers.

The point is that the default setup of Ubuntu and Windows is what's important to most people. My mother is a teacher in multimedia and she uses both WinXP and Ubuntu. She doesn't know what the kernel even is, let alone how to recompile it. So what she, and many others, are stuck with is the default configuration of her system. That's what matters. That's what I care about.

But sure, for tech-readers it may be interesting to see what you can get if you everything you absolutely can to get good results, including recompiling with optimized flags, configuring software or even change different parts of the system. I don't think that's what the author really aimed at, though.

Edited 2008-05-21 11:39 UTC

I'd like to reply here:
My test was not about how fast Ubuntu or Linux in gerenal CAN BE. It's how Ubuntu 8.04 perform against Windows XP SP3. I'd stick to its default setup with a couple of easy tweaks that can also be done on Windows. I don't even pretend to know the reasons behind the benchmark scores. I merely suggest what my limited knowledge help me with.

Well, improving performance and improving stability are completely different things, don't you think?


Of course I know how to customise linux. In fact, I never settle with the default options, but again, it's a test of Ubuntu Hardy as it is offered to the world against Windows XP SP3 as Microsoft did it.

Oh, thanks for taking your time and doing a great test, btw.

What stays longer in the end-users mind is the actual waiting time or perhaps the perceived speed.

How long do you have to watch the hourglass?

I agree with you on what WE the users want, but Ubuntu precisely fails at that. Responsiveness to the user is lower than with Windows' "convoluted" system.
Maybe desktop oriented distros should patch the scheduler to do what the users want it to do instead of being potentially useful for some freak background scientific calculations or Internet services.
Windows has a switch for Linux-style unresponsive mode(it's under computer advanced settings).
Firefox is slower in Linux, hell, firefox GUI is less responsive in Linux than in OpenBSD(!), and that's saying something since OpenBSD is a server oriented OS that seeks security over performance and Desktop features. You can blame the background Internet orifices, I mean services, that most certainly are enabled by default in Linux, but even if it came with a strong AI service, the GUI should always have a large enough chunk of the processor time reserved so as to never be unresponsive.
I will give up a 20% reduction in compilation time just to have a responsive GUI, any day.

You are argumenting your claim totally wrong. Spending more time doing task switches doesn't improve responsiveness, it only means the applications have even less CPU time overall for them. Imagine if you had two systems, one which spent 5% of its CPU time doing task switches and another one which spent 10% of its time doing the same; system 1 would leave 95% left for the applications and system 2 would leave 90% left for them, no matter how many task switches each one does.

The fact is, the less time the system spends doing application management instead of running the application the less time the apps have to respond to user input.

No matter how you put it, an increase in the number of task switches reduces the average time it takes before a task gets the cpu. This means it can react to new input with a shorter delay. That's responsiveness.

Sure, with less task switches the tasks get a bigger share of the CPU, but it gets its share in big chunks with a lot of time in between (this time is when the other tasks get their share). During that time it can't take user input, which results in a loss of responsiveness.

The only exception is a system with only one task, where less task switching improves responsiveness. But if you have only task, then you don't need task switches.

In my opinion loss of responsiveness is not acceptable under any circumstances ever. Sadly, major OSes thinks its completely OK, and in fact an excellent idea to regularly prevent the user from using his computer for up to a second (in bad cases) or even longer.

You do realize, I hope, that task switches are measured in milliseconds? Increasing the frequency of task switches will not get you a more responsive-to-human-input environment; other factors (such as scheduling algorithms) completely swamp that factor.

Nor is it reasonable to assume that an environment that runs slower must be switching tasks more frequently. It is much more likely to be spending more time running non-interactive tasks such as virus scans, handling system tasks such as network I/O, or dozens of other routine matters.

My personal experience on computers with both XP and Ubuntu installed has been that Ubuntu is noticeably more responsive, principally because XP spends so many cycles scanning for malware (using McAfee and Norton Defender). However, how you configure both environments will have a great impact on how responsive the environments "feel".

You should use what you like, of course.

My experience shows that Ubuntu 8.04 is more responsive than XP. That is one thing I notice when switching from XP to Ubuntu. While some heavy task running that make my processor really hot, I can switch applications and desktops easily, and do something with them.

It really surprised me really, because if that happened in XP, switching applications will give me white blank windows, and I have to wait until XP can redraw the windows.

Edited 2008-05-21 14:20 UTC

The multitasking test has the right conclusion!

From my experience with running applications (for example FE-Codes like ABAQUS or EXCITE), Linux is much more responsive on heavily loaded to overloaded systems than Windows XP. At least this is true for RHEL3.x and XP SP2.

When you run an EXCITE process on a 4 core Linux system and tell it to use 4 CPU's, you will notice some less responsiveness, but you can still work on the machine. Renice the EXCITE job, and you will notice almost no change in responsiveness to an unloaded system.

Do the same on XP, and the computer can barely run a web browser with decent responsiveness.

This is just my observation when I am working for my employer. I have no measurements to back that up with numbers, but it feels like Linux looses 10 - 20% of its responsiveness, and XP looses 50 - 70%. It is consistent through several Linux and Windows machines.

What I do now: I just run the job on a Windows machine with 3 CPUs, and loose some time through this.

When Windows grinds to a halt like that it's usually because of the different (and obviously dreadful) policy when it comes to memory caching and swapping. As soon as something is swapped to disk, you've basically lost the performance battle. Linux seems to excel here, by not swapping to disk so quickly.

At this very moment I'm running while (true) loop that takes as much cpu time as it can get, on an old single-core processor.
There is almost no loss of responsiveness. I can browse the web without noticing anything. Redrawing minimized explorer windows takes probably 0.2 seconds more. Minimizing and restoring Opera is as fast as normal.

I haven't tried this on Linux, but it can't possibly be better, because XP is so good in this matter.

Of course doing background tasks will take more time, but I don't care as long as I can do my foreground tasks.

Edited 2008-05-21 20:28 UTC