I just like the fact that I feel like I somehow initiated
the whole thing.
Greg
------- Forwarded Messages
> ------------- Begin Forwarded Message -------------
>
> To: Tom Arkwright <forjava@chessie-10.eng.sun.com>
> Subject: Re: Solaris VM performance
> Date: Sun, 07 Sep 1997 09:33:23 -0700
> From: Gregory Alan Bolcer <gbolcer@gambetta.ics.uci.edu>
>
> > Subject: Solaris VM performance
>
> Hi Tom!
> The humble opinion of a java programmer, this is the only
> number that counts:
>
> http://www.webfayre.com/pendragon/
> http://www.pendragon-software.com/pendragon/cm3/index.html
> http://www.pendragon-software.com/pendragon/cm3/runtest.html
>
> Everything else is just marketspeak. We use this as a benchmark
> across all our machines, operating systems, and JVMs, and we
> test it on every new one that comes out.
>
> Greg
>
>
>
> ------------- End Forwarded Message -------------
> To: Tom Arkwright <forjava@chessie-10.eng.sun.com>
> cc: endeavors-dev@gambetta.ics.uci.edu
> Subject: Re: Solaris VM performance : Discussion from java-internal
> Date: Sun, 07 Sep 1997 09:49:03 -0700
> From: Gregory Alan Bolcer <gbolcer@gambetta.ics.uci.edu>
>
> Hi Tom!
> I like numbers, thought you might be interested.
>
> Greg
>
>
> Caffeine Mark 3.0 comparisons
> Summary: PII-266 (2926), SGI-O2-180 (934), Ultra1200 (1319)
>
> Gateway E-3100, Pentium II 266MHz, 64MB RAM, Windows NT 4.0,
> Microsoft Internet Explorer 3.02 with the Microsoft Java
> SDK 2.0, ATI Rage 3D Graphics.
>
> Sieve 4196
> Loop 14341
> Logic 11715
> String 4995
> Float 7431
> Method 6171
> Graphics 1108
> Image 476
> Dialog 185
> CaffeineMark 2926
>
>
> SGI O2 180mhz R5000, 64MB RAM, Irix 6.3, SGI JDK 1.1.3 w/JIT
> appletviewer.
>
> Without JIT:
> Sieve 125
> Loop 117
> Logic 129
> String 117
> Floating-Point 117
> Method 121
> Image 54
> Graphics 601
> Dialog 29
> Compile (optional)
> AGC (optional)
> CaffeineMark 112
>
> With JIT:
> Sieve score = 623
> Loop score = 995
> Logic score = 1750
> String score = 731
> Float score = 766
> Method score = 1097
> Overall score = 934
>
> Sun Ultra2 200mhz, Creator graphics 128MB RAM, Solaris 2.5.1
>
> Without JIT:
> Sieve 113 229
> Loop 98 226
> Logic 97 209
> String 120 282
> Floating-Point 106 265
> Method 103 233
> Image 299 205
> Graphics 621 1181
> Dialog 130 75
> Compile
> AGC
> CaffeineMark 147 246
>
> With JIT:
> Sieve score = 1302 1310
> Loop score = 2061 2058
> Logic score = 3445 3440
> String score = 818 779
> Float score = 1286 1278
> Method score = 574 572
> Overall score = 1331 1319
>
>
>
-------------- Begin Forwarded Message -------------
Date: Mon, 8 Sep 1997 13:01:46 -0700 (PDT)
From: "Steve O'Neill" <soneill@jurassic.eng.sun.com>
Subject: Re: How our customers view JVM benchmarks...
To: stephen.oneill@Eng, forjava@chessie-10.eng.sun.com
Cc: butch.holmberg@Corp, phil.parkman@Eng, james.liu@Eng
MIME-Version: 1.0
Content-MD5: 4CugalMhTx3Gp7vKhRN0/Q==
Ah the gauntlet has been tossed. First if I was calling myself a Java Programmer
I would sure have to have a humble opinion. HA HA.
Seriously though... Caffeine Mark 3.0 is the reference here and this is what we
(Sun) need to address. This bench mark has become the de-facto standard in the
industry because no one has addressed the issue of spoofing (cheating) this
bench mark. And just as a side note with a little fiddling of the OS I ran
26000+ just last week. I don't know who is the right person to start addressing
this with the customers but some one needs too!
Further explanation:
Caffeine Mark 3.0:
This mark is at best an indicator of CPU and how well you have optimized the
byte code. This test actually runs 9 separate tests. But it runs them twice
each. The first pass is to load, JIT, optimize everything and then the second
pass is where the result is produced. So what is really happening is the load,
JIT and memory allocation time is not counted and you can pretty much do
anything you want including run pre compiled code. This bench mark can be
seriously spoofed and there are other groups (Kumar's) who have been able to get
quite absurd numbers out of this mark. Myself can pretty much give you what ever
you would like by changing one OS #define.
A proper bench mark harness will have the characteristic of multiple tests where
if you try to tune for one you screw up some of the other tests. If you
accomplish this then what you get is an accurate bench mark that really is
indicative of the true performance of the target. Currently there is an effort
inside of Sun to push a harness that we have put together into SPEC.
So I guess to put this plainly the Caffeine Mark 3.0 bench mark is BS. The real
problem is just as he has stated. Which is he is a Java Programmer which really
means he doesn't understand what performance is about and we need to educate
these people!!!!
My $ 0.02
Ta...
steve
\|/ ____ \|/
@~/ oO \~@
/_( \__/ )_\
\_U__/
Stephen R. O'Neill
Performance Architect
Java Computing
Enterprise Engineering
Sun Microsystems Computer Corp.
(415) 786-6932
"Don't pay attention to the Squishy noises"
"The law of Probability Dispersal decrees that whatever it is that
hits the fan will not be evenly distributed."
- ------------- End Forwarded Message -------------
------- Message 2
Date: Mon, 12 May 1997 10:46:30 -0500 (CDT)
From: Philip Helsel Sun SE-San Antonio 210-4904786 <philip.helsel@Central>
Subject: Solaris VM performance : Discussion from java-internal
To: aces.java@Sun.COM
MIME-Version: 1.0
Content-MD5: NErFBP6T1wMFjxbOCZfJ0A==
Aces:
FYI.
PeH
- ------------- Begin Forwarded Message -------------
Date: Sat, 10 May 1997 20:48:01 -0700
From: Mario.Wolczko@Eng (Mario Wolczko)
To: ronk@shadows.Eng.Sun.COM
CC: Stephanie.Vargo@Eng, java-internal@shadows.Eng.Sun.COM
Subject: Re: MSFT's Java VM
> DEC (remember them?) got 15 very good years from the Vax/VMS combination
> by pitching it as a Fortran engine. We could have gotten similar mileage
> from positioning SPARC/Solaris as a Java engine.
>
> A bit late now.
The performance shortfall of Java on Solaris (indeed all of Sun's Java
implementations) is being, and will be, fixed.
Sun is fortunate to employ a large number, perhaps even the majority,
of the world's VM implementation experts. Some are at JavaSoft
(including the group that came along with the acquisition of
Animorphic), some in Sun Labs, others are at SunSoft and elsewhere.
Collectively, these folks have decades of VM experience.
Sun woke up rather late to the fact that JVM performance is important,
and other companies managed to get ahead in the race. However, I am
confident that this will change in the next year. Watch out for
really impressive performance improvements.
However, in concert with the engineering, we need to be spreading the
right message. Customers need to be told that we are aware of the
problems, and that we are fixing them. We must also make sure that we
counter hype from our competitors. For example, it's often stated
that Microsoft's VM or some other VM is 3, 10 or some other factor
faster than ours. While this is sometimes true on specious benchmarks
such as Caffeinemark, there is desperately little empirical data
available for real applications. An example: one major customer told
us that their app was 10 times faster with Microsoft's VM on NT.
However, when we dug deeper, we found that they had never run their
app on NT -- they were extrapolating from small benchmark programs
with no justification that the benchmark was a fair characterization
of their app.
At SunSoft, we are bringing in customer apps to evaluate these claims
and to assist in JVM tuning. We already have one major app (0.5M
lines of Java) in house, and we're getting another this week. We want
more however, so if you know of customers who are developing Java apps
and believe they have JVM-related performance problems, please get in
touch with me. We're not going to fix their problem right away, but
we can use their app to learn more about how customers are using Java.
The deal we can strike with them is that we get to exercise our JVM on
their code, and they are assured that their app will influence the
development and tuning of future JVMs from Sun.
Mario Wolczko
Project Lead, SunSoft JVM group.
- ------------- End Forwarded Message -------------
- -------------------------------------------------------------------
_/_/_/ _/ _/ _/ _/ Philip E. Helsel
_/ _/ _/ _/_/ _/ Systems Engineer SMCC.
_/_/_/ _/ _/ _/ _/ _/ 13750 US 281 N., Suite 605
_/ _/ _/ _/ _/_/ San Antonio, Texas 78232
_/_/_/ _/_/_/ _/ _/ philip.helsel@Central.Sun.COM
210-490-4786 (phone direct)
M I C R O S Y S T E M S ...ext 303 (voice mail)
210-490-8356 (fax)
"In a world without Fences...Who needs Gates" Java(tm)
From a tee-shirt seen at JavaOne
- --------------------------------------------------------------------
- ------------- End Forwarded Message -------------
------- Message 3
- ------- Forwarded Message
Sun accused of cheating Java benchmarks
(http://www.zdnet.com/zdnn/content/zduk/1105/206023.html)
Sun defends Java benchmark practices (http://www.zdnet.com/pcweek/news/1103/06mcheat.html)
- - -----Original Message-----
From: John Clingan [SMTP:john.clingan@West.Sun.COM]
Sent: Tuesday, October 21, 1997 4:54 PM
To: ocjug@sunkist.West.Sun.COM; mtjug@majordomo.hughes.net
Subject: SUN'S SOLARIS RUNS JAVA APPLICATIONS 50 PERCENT FASTER THAN WINDOWS NT
FYI, I typically don't put out these types of (marketing) press releases to
this list, but note the second paragraph on JDK 1.1.4 performance improvements
under Solaris and the last paragraph on availability. Thought the Solaris
users on this list might find this informative.
- - ---------------
SUN'S SOLARIS RUNS JAVA APPLICATIONS 50 PERCENT FASTER
THAN WINDOWS NT
JDK Optimized For Solaris Available Only At
http://www.sun.com/solaris/java
Palo Alto, Calif., October 20, 1997 - Sun Microsystems, Inc.
today announced that its new Web-enhanced SolarisTM operating
environment delivers the world's fastest JavaTM technology performance.
The new Solaris record, 50 percent faster than Windows NT, is a result
of benchmarks conducted by Sun using CaffeineMark 3.0 (measures
the speed of Java executions), and Solaris 2.6 software.
The power of Java technology is accessed through the preview release
of Java Development Kit (JDKTM) 1.1.4 for Solaris, a set of building
blocks that includes basic development tools and a rich set of class
libraries. Version 1.1.4 includes enhancements to the Java Virtual
Machine that increases Java application performance up to three times
faster than before.
The new Web-enhanced Solaris 2.6, with integrated Java technology,
provides the best foundation for companies to run Java applications
at significantly higher speeds than Windows NT. Customers deploying
Java applications on Solaris can leverage performance gains while
reaping the rewards of Java's `Write Once, Run Anywhere(TM)' vision
that reduces application development cycles. Solaris operating
environment provides the infrastructure for the WebTone, the 21st
century equivalent of today's dialtone, and addresses customer needs
in intranet, Internet, enterprise and power desktop computing.
"Sun is dedicated to delivering faster Java performance on Solaris
than any other operating system," said Steve MacKay, vice president
and general manager of the Solaris Products Group, Sun Microsystems.
"Fast Java performance means improved productivity for customers
who choose Solaris to deliver the WebTone."
Solaris Test Configuration
" Web Enhanced Solaris operating environment
" SunTM UltraTM EnterpriseTM 30
" 1 x 300 MHz UltraSPARC
" 500 MB RAM
" Seagate Barracuda ST32171WC (2.1GB)
" FFB2 graphics card
Availability
Download the JDK optimized for performance on Solaris at
http://www.sun.com/solaris/java. Visit this page for additional
information about Java on Solaris, the new Web-enhanced Solaris,
the latest Java Development Kit/Just-In-Time compiler, white papers
and other valuable resource.
_____________________________________________________________
______
/_____/\
/____ \\ \ John Clingan
/_____\ \\ / Sun Microsystems
/_____/ \/ / / 222 N. Sepulvida, 18th Floor
/_____/ / \//\ El Segundo, CA 90245
\_____\//\ / / Phone 310/607-2404
\_____/ / /\ / Email: John.Clingan@West.Sun.COM
\_____/ \\ \
\_____\ \\
\_____\/
- ------- End of Forwarded Message
[Image]
[Image]
Sun says it "optimized'' Java tests
[Image]3:27 p.m. EST (2027 GMT) November 7, 1997
SAN FRANCISCO (Wired) - In response to a claim that it
manipulated the Solaris operating system code to maximize
its score on an independent Java benchmark test, Sun said
Thursday that, yes, of course it did.
"We'll try to make our implementation as fast as possible -
anyone would do that,'' said Brian Croll, director of
marketing for Sun Solaris products, who added, "we do know
that Microsoft optimized around it too.''
Sun and its competitor, Microsoft, are waging legal and
technical battles over the use of Java, a computer language
aimed at making computer applicattions entirely portable and
operable over a network.
Pendragon Software Wednesday accused Sun of adjusting its
Solaris compilers for the CaffeineMark software, which
Pendragon developed as a neutral test for how Java
applications will perform on different platforms.
Croll said that Sun did not use code from the CaffeineMark
software to optimize the Solaris results.
"If it turns out that CaffeineMark's instrumentation is a
little off, then let's take a look at it for the next round
of benchmarks,'' he said.
The idea behind benchmarks is that they will reflect the
performance of a wide range of applications on a platform,
not just one implementation.
But in the case of Java, there are very few applications
that exist - much less in widespread use - and so the
benchmarks are more like pre-tests than real-world
indicators.
As vendors communicate more with benchmark software
designers, the tests will more accurately represent the true
performance of a platform. So as Java applications evolve,
so too will benchmarks.
"Historically, the industry looks at benchmarks and comes
out with another round of specs that are more accurate,''
said Croll. "There is some evolution that needs to happen,
and that's the role of the writer, not the vendor.''
Reuters Ltd. All rights reserved
FOX News Network 1997. All rights reserved.
comments@foxnews.com
About FOX News Internet
------- End of Forwarded Messages