Re: It is almost certain now, INTEL will have 64bit x86 !!

From: Rick Jones (foo_at_bar.baz.invalid)
Date: 02/26/04 

Date: Thu, 26 Feb 2004 19:54:45 GMT

Andrew Harrison <andrew_._remove_harrison@su_n.com> wrote:
> Rick Jones wrote:
>> The tradeoffs between CPU/Memory/IO sound rather like the SunFire
>> systems.

> No they are not, A SunFire system can have 72 CPU's, 576 GB
> or RAM and 18 I/O controllers without any tradeoffs.

Perhaps we are talking about different variants of tradeoffs.

What about the "MaxCPU" stuff - if I've read the stuff I've been able
to find thusfar correctly, those go into IO slots. Doesn't that mean
that if one wants Maximum CPU count on the 15K one has to give-up I/O?

>> If they can actually then drive that I/O out the PCI slots.

> Of course but then as you would also know Sun has demonstrated
> 12.5 GB/s on a table scan from an Oracle DBMS sustained through
> the older I/O subsystem for the F15K what can you do

The E15K specs *** on www.sun.com claims the IO is "up to 21.6 GB/s
sustained"

http://www.sun.com/servers/highend/sunfire15k/specs.xml

> And its sustained I/O that we quote not a marketing BS number based
> on the number of slots x the speed of the slots which is the tried
> and tested faire for the HP marketing dept as you also know.

In that same data *** it also says:

  "Up to 72 I/O hot-swappable PCI slots; 36 slots are 66 MHz;36 slots
   are 33 MHz;"

But doesn't say how many PCI busses or if those are 32 or 64-bit PCI.

The "18 channels" also mentioned are I belive the I/O bays, which I
guess is consistent with the "up to 17 MaxCPU boards" as one would
need to leave one I/O bay.

Interestingly enough, if I assume that each of those PCI slots are
independent busses, each 64-bits wide, and one were to run a PCI bus
to 100% of the fequency x width that comes-out at 27 GB/s. (64x66
being 1/2 GB/s, 32x66 being 0.25 GB/s) That is rather close to the
claim of "up to 21.6 GB/s sustained" which would be an assumption of
running at 80% - and 21.6/27 is exactly 0.80 which I find remarkable.

Sure sounds like some-one in marketing was adding number and speed of
slots.

And if indeed, those are _not_ 72 independent PCI busses, the person
adding the number and speed of slots to get the 21.6 GB/s figure
forgot to take that into account...

Going further, I managed to find the E15K Overview guide:

http://www.sun.com/products-n-solutions/hardware/docs/pdf/806-3509-12.pdf

and the way it is written, it sounds like to access memory on a
CPU/Memory board, there has to be a CPU present for each 8 DIMMs, (the
memory controller being on the CPU and all...) which suggests you
cannot have a maximum memory configuration without populating each of
the CPU/Memory cells with CPUs whether you need the CPUs or not.

 "The CPU/Memory board holds four CPUs. Each CPU has an associated
  memory subsystem of eight DIMMs, so memory bandwidth and capacity
  are both scaled up as CPUs are added."

In discussing the I/O it says:

 "The common Sun Fire 15K/12K systems hot-swap PCI assembly
  architecture has two I/O controllers. Each controller provides one
  66/33-MHz PCI (peripheral component interconnect) bus and one 33-MHz
  PCI bus for a total of two of each speed on the I/O board.
  Therefore, each I/O assembly has four hot-swap component PCI
  slots. A Sun Fire I/O board has a 2.4 Gbyte per second connection to
  the rest of the system."

Which suggests that there are two PCI busses per I/O board, each with
two slots, which means there are only 36 PCI busses not the 72 I
assumed above. That seems to call into question the "21.6 GB/s
sustained" claim from the specs page.

It also suggests that the table scan number you quoted was pushing the
I/O to the limit. A table scan is reading from mass storage right?
That means it was DMA write performance, which will be less affected
by memory latencies than DMA read would be.

As for the newer 25K, the brief description at
http://www.sun.com/servers/highend/ also mentions MaxCPU boards.
Calling something "PCI+" is interesting - I guess that is a term Sun
concocted. The brief data *** at:

http://www.sun.com/servers/highend/sunfire_e25k/sunfire_e25k_data***.pdf

suggests that the I/O cells have been tweaked to have three 66 MHz
slots and one 33 MHz slot as it says:

  "Up to 72 hot-swappable PCI+I/O slots; 54 slots are 66 MHz;18 slots
   are 33 MHz"

But it still does not say anything about the number of PCI busses. It
could just as easily be they simply shifted a slot from the 33 MHz bus
to the 66 MHz bus, in which case the total I/O through the slots is no
better than the 15K. Or they could have added another 66 MHz bus.
I'd love a pointer to a URL that clears that up as the 15K equivalent
HW docs don't seem to be up yet for the 25K.

If indeed there isn't another 66 MHz bus on those I/O boards on the
25K, it might be 'PCI+' but the end-result would be doubleplusthesame.

Inriguingly, for sustained I/O on the 25K, the data*** says:

  "Up to 25.2-GB/sec. sustained"

Which still sounds like counting slots and frequencies and widths as
the sums of those is 37.5 and 80% of that is indeed 25.2 GB/s...

And given that the memory controller(s) is(are) still on the
UltraSPARC-IV chip, along with the two CPUs, the bit about needing to
add CPUs to add memory would seem to still apply.

So, no, I am not intimately familiar with the 8400, but the high-end
SunFire servers certainly seem to have their share of tradeoffs.

rick jones 

-- 
firebug n, the idiot who tosses a lit cigarette out his car window
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to raj in cup.hp.com  but NOT BOTH...
Quantcast