(Follow this link to go back to the main SGI page, and this link to go back to the previous part.)
The machine coverage of NetBSD on sgi increased to reach two of the oldest MIPS-based SGI machines, the Personal Iris 4D systems (IP6 and IP10), again thanks to the hard work of Steve Rumble.
Enable Personal IRIS 4D/20 and 4D/25 support:
- Adapt int(4) to handle the INT1 chip
- Move generic rtc clocks out of hpc/ and into dev/
- Handle the very strangely wired eeprom and other bits in arcemu
- Sprinkle MACH_SGI_IP6 as necessary
- Enable IP6/IP10 devices in GENERIC32_IP12. Yes, the naming is poor but
there's no winning with kernel/hw compatibility on sgimips...
Tested on my 4D/25. Doesn't (appear to) break macallan@'s IP22.
A few days later, Michael Lorenz contributed X server drivers for both the XL (newport) graphics for Indy, and the O2 frame buffer.
At about the same time work occurred on the O2 display in NetBSD, Joel Sing was also working on the OpenBSD side and added console acceleration on february 24th.
Add support for hardware acceleration to gbe(4). This provides an accelerated framebuffer for the console on SGI O2 workstations. X is still supported via wsfb(4) by switching back to the unaccelerated linear framebuffer mode. Some hardware details and magic numbers from NetBSD's crmfb(4) driver.
Date: Sun, 12 Apr 2009 18:25:18 +0000
From: Miod Vallat
To: Jasper Lievisse Adriaanse, Joel Sing , Martin Reindl
Cc: Theo de Raadt
Subject: Octane status
I have commited all bits necessary to get RAMDISK kernels to run on
ip30 (except for the kernel configuration files).
It's a good thing I got myself a second Octane, because the new one has
an R12000 cpu, a version 4 Xbridge (thus allowing me to test the >= 4
codepath), and exposed bugs the other one did not.
So after enough tinkering, I got both machines to run. There are kluges
all over ioc.c though.
I have changed the locators of ioc subdevices, to not use `irq' as the
second locator - these are not really irq but subdevice numbers, so the
config stanzas become:
com0 at ioc? base 0x00020178 dev 1
com1 at ioc? base 0x00020170 dev 2
I am considering removing the ioc locators from the kernel configuration
file and have ioc provide them itself from a table (kinda like obio on
many other architectures) to prevent people from playing with locators.
But that's not important at this point.
Now, the todolist is:
- in parallel:
- write an onboard Ethernet driver. I think Joel can be bribed to
work on this (-:
- get o200 to run further (I need to fix the widget enumeration logic,
which is wrong on this machine, and then write the pci xhub code).
That's what I'll be working on ASAP.
- once we have a machine with Ethernet (either Octane onboard or o200
with an extra PCI card), introduce distinct kernel configuration files
(GENERIC-IP*, RAMDISK-IP*), and discuss this to explain why this
is necessary. Update the distrib/ machinery to use *-IP32 for O2, do
not build Octane media yet.
- similar changes to the disk bootblocks: they will need different link
address. Devise a proper machinery to get the bootblocks with
different names in /usr/mdec, alter installation media to use ip32
bootloader.
- probably add a sysctl to the kernel to report what kind of boot and
kernel it needs (IP27/IP30/IP32). Start using this in the installation
media, with a small sysctl retrieval tool a la ``disknames''. Or maybe
a proper regexp on the kernel name in dmesg would be enough (since it
will be RAMDISK-IP##).
- once we have installation media able to work on O2, Octane (and
probably Origin 200 by this time), celebrate, then start working on
video and glass console... and on SMP... etc, etc. We're not there
yet.
[...]
The next day, I also had made some progress on Origin 200, which was however behind.
From: Miod Vallat
To: Jasper Lievisse Adriaanse
Cc: Joel Sing, Martin Reindl, Theo de Raadt
Subject: Re: Octane status
[...]
> > - get o200 to run further (I need to fix the widget enumeration logic,
> > which is wrong on this machine, and then write the pci xhub code).
> > That's what I'll be working on ASAP.
> I'll happily test diffs on my o200's.
Well I'm slowly getting there, here is what I got minutes ago:
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2009 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.5-current (RAMDISK-IP27) #10: Mon Apr 13 13:21:34 GMT 2009
miod@santoire.gentiane.org:/usr/src/sys/arch/sgi/compile/RAMDISK-IP27
real mem = 1073741824 (1024MB)
rsvd mem = 12591104 (12MB)
avail mem = 1002553344 (956MB)
mainbus0 at root
cpu0 at mainbus0: MIPS R10000 CPU rev 3.4 225 MHz with R10000 FPU rev 0.0
cpu0: cache L1-I 32KB D 32KB 2 way, L2 2048KB 2 way
clock0 at mainbus0: ticker on int5 using count register
node0 at mainbus0
mboard nasid 0 slot 1 not configured
mplane nasid 0 slot 48 not configured
xbridge0 at node0 nasid 0 slot 1 revision 4
pci0 at xbridge0 bus 0
isp0 at pci0 dev 0 function 0 "QLogic ISP1020" rev 0x05: couldn't
establish interrupt at irq 0
isp1 at pci0 dev 1 function 0 "QLogic ISP1020" rev 0x05: couldn't
establish interrupt at irq 1
ioc0 at pci0 dev 2 function 0 "SGI IOC3" rev 0x01: failed to establish
superio interrupt!
rd0: fixed, 8192 blocks
boot device: lookup 'sd0a' failed.
root on rd0a swap on rd0b dump on rd0b
However, I have needed to change many things in sgi/xbow/ (basically to
extend the code from an ip30 view of the world, with a single set of
widgets and a friendly xbow arbiter, to an ip27/ip35 view of the world,
with multiple widget locations and different device tree anchorage, see
the ``node'' bus in the dmesg above).
I need to work on interrupt support for ip27 and hopefully I'll reach
single-user mode soon. Then I'll commit the o200 parts.
In the meantime, OpenBSD users had noticed the Octane activity and asked for more information.
Date: Wed, 15 Apr 2009 00:44:52 +0000 From: Naoki Hamada To: openbsd-sgi Subject: octane support Hello guys, I recently got an octane and want to make it a bsd machine. I am very happy to find there are commits around IP30. I tried to generate a kernel but there lack some IP30 parts (config file, for example) yet. If they are commited, I would willingly test them and hopefully improve them. Naoki Hamada
To which I replied:
Date: Wed, 15 Apr 2009 18:59:21 +0000
From: Miod Vallat
To: Naoki Hamada
Cc: sgi@openbsd.org
Subject: Re: octane support
> I am very happy to find there are commits around IP30. I tried to
> generate a kernel but there lack some IP30 parts (config file, for
> example) yet. If they are commited, I would willingly test them
> and hopefully improve them.
The lack of the kernel configuration files is intentional (-: They will
appear when the machines are a bit more usable.
At the moment, Octane will run using a serial console, and the onboard
SCSI controllers. However, the onboard interface is not supported yet,
and should you be the happy owner of a PCI Cardcage extension, or a PCI
Shoehorn XIO card, their PCI devices will not configure correctly yet.
As long as the machine can not talk over the network, running BSD on it
is quite useless.
Please give us a bit more time to make progress on this!
Miod
PS: and before people ask, here is the log of bsd.rd booting on one of
my Octanes:
Running power-on diagnostics...
System Maintenance Menu
1) Start System
2) Install System Software
3) Run Diagnostics
4) Recover System
5) Enter Command Monitor
Option? 5
Command Monitor. Type "exit" to return to the menu.
>> bootp()/bsd.ip30
Setting $netaddr to 10.0.1.211 (from server )
Obtaining /bsd.ip30 from server
5859248+484952 entry: 0xa800000020020000
ARCS64 Firmware Version 64.0
Found SGI-IP30, setting up.
Initial setup done, switching console.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2009 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.5-current (RAMDISK-IP30) #75: Wed Apr 15 18:24:43 GMT 2009
miod@santoire.gentiane.org:/usr/src/sys/arch/sgi/compile/RAMDISK-IP30
real mem = 1073741824 (1024MB)
rsvd mem = 1064960 (1MB)
avail mem = 1013821440 (966MB)
mainbus0 at root
cpu0 at mainbus0: MIPS R12000 CPU rev 2.3 300 MHz with R10000 FPU rev 0.0
cpu0: cache L1-I 32KB D 32KB 2 way, L2 2048KB 2 way
clock0 at mainbus0: ticker on int5 using count register
xbow0 at mainbus0: XBow revision 4
xheart0 at xbow0 widget 8: Heart revision 6
onewire0 at xheart0
owserial0 at onewire0 "16kb EPROM" sn 0000003c3d84
owserial0: "PM10300MHZ" serial 030-1355-001
"ImpactSR" revision 2 at xbow0 widget 12 not configured
xbridge0 at xbow0 widget 15: Bridge revision 4
pci0 at xbridge0 bus 0
isp0 at pci0 dev 0 function 0 "QLogic ISP1020" rev 0x05: irq 0
isp0: invalid NVRAM header
scsibus0 at isp0: 16 targets, initiator 7
sd0 at scsibus0 targ 1 lun 0: <QUANTUM, ATLAS IV 9 SCA, 0707> SCSI3 0/direct
+fixed
sd0: 8683MB, 512 bytes/sec, 17783249 sec total
isp1 at pci0 dev 1 function 0 "QLogic ISP1020" rev 0x05: irq 1
isp1: invalid NVRAM header
scsibus1 at isp1: 16 targets, initiator 7
ioc0 at pci0 dev 2 function 0 "SGI IOC3" rev 0x01: superio irq 4, ethernet irq 2
onewire1 at ioc0
owmac0 at onewire1 "1kb EPROM" sn 0000029e542f
owmac0: Ethernet Address 08:00:69:13:4b:f8
owserial1 at onewire1 "16kb EPROM" sn 000000348d70
owserial1: "FP1" serial 030-0891-003
owserial2 at onewire1 "16kb EPROM" sn 000000631bf1
owserial2: "PWR.SPPLY.ER" serial 060-0035-003
com0 at ioc0 base 0x00020178 dev 1: ns16550a, 16 byte fifo
com0: console
com1 at ioc0 base 0x00020170 dev 2: ns16550a, 16 byte fifo
dsrtc0 at ioc0: DS1687
"SGI Rad1" rev 0xc0 at pci0 dev 3 function 0 not configured
rd0: fixed, 8192 blocks
boot device: sd0
root on rd0a swap on rd0b dump on rd0b
WARNING: clock time much less than file system time
WARNING: using file system time
WARNING: CHECK AND RESET THE DATE!
erase ^?, werase ^W, kill ^U, intr ^C, status ^T
(I)nstall, (U)pgrade or (S)hell? S
# ls
.profile etc install.sub sbin usr
bin install mnt tmp var
dev install.md mnt2 upgrade
#
You might remember the friend I had mentioned got two Fuel systems in march 2007. As a joke, he had dared me several times to make OpenBSD run on them. As work on the Origin 200 and the Octane was slowly making progress, I would have liked to also test it on the IP35 family, in order to try to understand the differences between IP27 and IP35, and make sure the way I would design or write my code to support IP27 would not get in the way of a possible future IP35 support. I asked him whether he would accept to lend me one of his Fuel systems, and he gladly did (telling me "don't you dare to give it back until you have a working OpenBSD port".)
Date: Wed, 15 Apr 2009 20:18:54 +0000 From: Miod Vallat To: Jean-Marc Harang Subject: Fuel Quand est-ce que je peux passer t'emprunter une Fuel ? Pas besoin de disque dedans. [When can I drop by and borrow a Fuel? I don't need a disk in it.] C'est un peu tôt compte tenu de l'état encore rustique du support Octane et Origin 200, mais en fait jouer avec maintenant me permettrait d'avoir plus d'informations pour effectuer des choix de conception (et en particulier dans quelle mesure Fuel est une Origin 200 dans une configuration particulière, où s'il y a suffisamment de différences pour justifier une vie séparée). [It is a bit early given the current rustic state of the Octane and Origin 200 support, but being able to tinker with it now would let me gather more information in order to make good design choices (and especially figure out how much resemblance there is between Fuel and Origin 200, and if there are enough differences to warrant separate code bases.)]
Looking at the Linux code, the Octane and O2 onboard Ethernet controllers, albeit different, were very similar on many aspects. I tried to adapt the existing OpenBSD O2 Ethernet driver into running on the Octane, but it did not work yet. Not knowing whether I would spend more time on it, as there were so many other things to work on and I could use PCI Ethernet cards in the Origin 200, I shared the code with Joel Sing and Jasper Lievisse Adriaanse in case they would have time to help.
Date: Thu, 16 Apr 2009 20:06:31 +0000 From: Miod Vallat To: Joel Sing, Jasper Lievisse Adriaanse Subject: Octane Ethernet boilerplate I have had a look at the linux code for this beast, and came up with this. There are some similarities with if_mec on O2, but MEC is probably a later design correcting a few annoying problems in EF (such as having to get TX interrupts after each TX ring because the TX errors are reported as interrupts and not as per-descriptor status information). So the following code is if_mec with s/mec/ef/ and updated to play with the EF registers. However, don't get your hopes up: this does not work. What works: - ifconfig. - mii access (but this is trivial). What doesn't work: - everything else. Actually, I can't get the chip to interrupt. I have added a dump of the interrupt registers in the watchdog timeout, and it always remain stuck at zero. Things to consider: I did not port the linux code which looks for ssram and sets bits in the MCR register. This probably matters, and the SSRAM might be used as internal work memory, I don't know. So this is one of the first things to try. Then the ring buffer managment might be wrong as well. But this should provide a good starting point for anyone wanting to get this further. All the tedious task of writing the register defines and the function stubs is done (-: Miod PS: Configuration stanzas: ef* at ioc? icsphy* at mii? # that's what I have here nsphy* at mii? # that's what the Linux code suggests ukphy* at mii? PPS: I'll probably commit the ioc.c change soon, it cleans things up and is not intrusive. [...]
I spent the next few days trying to boot a kernel on the Fuel. The first few tries failed miserably, because the address my kernel was linked at was too low in memory and would overlap the ARCS reserved area (which is twice larger on IP35 systems than on IP27 systems.) After some trial-and-error I eventually found an address which would let ARCS accept my kernel image (I was attempting to boot directly from the ARCS firmware, over the network, so there were no debug functionalities available at this state until I could have it load my own code.)
The first successful kernel load occurred on april 19th.
It was mentioned on my next status report on the 20th.
Date: Mon, 20 Apr 2009 20:15:22 +0000
From: Miod Vallat
To: Jasper Lievisse Adriaanse, Joel Sing, Martin Reindl
Cc: Theo de Raadt
Subject: latest Octane status
Ok, here is the current status report of IP27/IP30/IP35 support:
* IP30 (Octane)
- jsing@ and I are noticing that isp(4) freaks out eventually, after
enough I/O, and preferrably under load. I am suspecting a bug in the
interrupt code which would cause reentrancy or spl not being honored
correctly.
- jsing@ is working on the on-board Ethernet driver; he's been able to
get interrupts from it, which is much better than my efforts so far,
so I'm not currently working in that area.
* IP27 (Origin 200/2000, Onyx 2)
- isp(4) does not work because it would need to do 64 bit DMA, which the
chip might not even be capable of; I have started to work on the iommu
part of the PCI bridge, which would allow isp(4) to work as badly as
on Octane (and perhaps better due to different interrupt code).
- devices plugged in the PCI slots are correctly recognized, but don't
have any resources assigned to them, as the PROM won't initialize
them; I need to work on this and see if this can converge with
kettenis@ plans.
* IP35 (Fuel, Tezro, Origin 300/3000, Onyx 300/350/4)
- I have been able to borrow a 600MHz R14000 model from a friend for a
while; after I figured out what address to link the kernel at, I've
been able to play a bit. It will be possible to use the same kernel
for IP27 and IP35, and the kernel is able to tell them apart (kind of
a kluge but reliable).
- The main difference between IP27 and IP35, apart from faster
components, is that the cpu board logic has been extended to support
four processors instead of two. Oh, and it has an USB controller
onboard too (-:
- The isp chip in this one is a high-end one (12160) and does not seem
to work yet, could be either a DMA mapping bug or lack of decent
support for this flavour in the driver. I suspect the former, and the
work on iommu will answer this question.
- No surprise with the PCI slots, their resources aren't mapped either,
so it needs the same work as IP27.
- But the major hurdle (which is puzzling the Linux guys too) is that we
don't get interrupts. I believe there is a register which needs an
explicit write to unlock remote writes to the node interrupt register.
And of course there is also the possibility of an incorrect
computation of that register address as seen from the bus, but I plan
to sort it out soon by trying the ~ 2**24 valid combinations with the
high resolution timer interrupt (-:
So, this still needs lots of work before one of these machines is
usable. But we're slowly making progress there.
Miod
PS: bootlog on the Fuel, to tease you guys:
Command Monitor. Type "exit" to return to the menu.
>> bootp()/bsd.ip27
Setting $netaddr to 10.0.1.212 (from server )
Obtaining bsd.ip27 from server
6103616+485152 entry: 0xa800000000040000
ARCS64 Firmware Version 64.0
Found SGI-IP35, setting up.
config @0x9600000000030000
magic 0xbeedbabe version 0
console 0x920000000f820178 baud 9600
Machine is in M mode.
Region present 0x1.
Calias size 0x2.
board type 11 slot 0 nasid 0 nic 0x553a318d components 4
hub widget 0 port -1 flag 0 speed 200MHz
memory 1024MB, select 0
bank 1 256MB
bank 2 256MB
bank 3 256MB
bank 4 256MB
memory from 0x0 to 0x4000000 (64 MB)
memory from 0x8000000 to 0xc000000 (64 MB)
memory from 0x10000000 to 0x14000000 (64 MB)
memory from 0x18000000 to 0x1c000000 (64 MB)
memory from 0x20000000 to 0x24000000 (64 MB)
memory from 0x28000000 to 0x2c000000 (64 MB)
memory from 0x30000000 to 0x34000000 (64 MB)
memory from 0x38000000 to 0x3c000000 (64 MB)
memory from 0x40000000 to 0x44000000 (64 MB)
memory from 0x48000000 to 0x4c000000 (64 MB)
memory from 0x50000000 to 0x54000000 (64 MB)
memory from 0x58000000 to 0x5c000000 (64 MB)
memory from 0x60000000 to 0x64000000 (64 MB)
memory from 0x68000000 to 0x6c000000 (64 MB)
memory from 0x70000000 to 0x74000000 (64 MB)
memory from 0x78000000 to 0x7c000000 (64 MB)
component widget 0 type 17
cpu type f24 600Mhz cache 4MB speed 300Mhz
board type 42 slot 0 nasid 0 nic 0xffffffffffffffff components 1
component widget 0 type 4
board type 54 slot d nasid 0 nic 0xffffffffffffffff components 1
component widget 13 type 10
board type 71 slot e nasid 0 nic 0x553a318d components 2
component widget 14 type 5
component widget 14 type 0
board type 71 slot f nasid 0 nic 0x553a318d components 4
component widget 15 type 5
component widget 15 type 37
component widget 15 type 6
component widget 15 type 34
Initial setup done, switching console.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2009 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.5-current (RAMDISK-IP27) #123: Mon Apr 20 20:09:27 GMT 2009
miod@santoire.gentiane.org:/usr/src/sys/arch/sgi/compile/RAMDISK-IP27
real mem = 1073741824 (1024MB)
rsvd mem = 47194112 (45MB)
avail mem = 1001119744 (954MB)
mainbus0 at root
cpu0 at mainbus0: MIPS R14000 CPU rev 2.4 600 MHz with Unknown FPU type (0x94) rev 7.5
cpu0: cache L1-I 32KB D 32KB 2 way, L2 4096KB 2 way
clock0 at mainbus0: ticker on int5 using count register
xbow0 at mainbus0: XXBow revision 2
"Odyssey" revision 11 at xbow0 widget 13 not configured
xbridge0 at xbow0 widget 14: XBridge revision 2
pci0 at xbridge0 bus 0
fxp0 at pci0 dev 2 function 0 "Intel 8255x" rev 0x0c: can't map i/o space
xbridge1 at xbow0 widget 15: XBridge revision 2
pci1 at xbridge1 bus 0
"QLogic ISP12160" rev 0x06 at pci1 dev 1 function 0 not configured
ioc0 at pci1 dev 4 function 0 "SGI IOC3" rev 0x01
onewire0 at ioc0
ioc0: superio irq 6, ethernet irq 4
com0 at ioc0 base 0x00020178 dev 1: ns16550a, 16 byte fifo
com0: console
com1 at ioc0 base 0x00020170 dev 2: ns16550a, 16 byte fifo
ohci0 at pci1 dev 5 function 0 "Opti 82C861" rev 0x10: irq 5, version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0 "Opti OHCI root hub" rev 1.00/1.00 addr 1
rd0: fixed, 8192 blocks
boot device: lookup 'sd0a' failed.
root on rd0a swap on rd0b dump on rd0b
WARNING: No TOD clock, believing file system.
WARNING: CHECK AND RESET THE DATE!
erase ^?, werase
and there it stops because the tx fifo is full and the interrupt is not
received. The fxp in pci0 is a card I put in one of the PCI slots of the
machine; I need to change the enumeration order so that the onboard
devices (i.e. pci1) attach first, BTW.
I have no idea why the FPU type is such a strange value, BTW.
Early may, I had made some progress on the Origin and Fuel front.
Date: Sun, 3 May 2009 20:27:13 +0000
From: Miod Vallat
To: Joel Sing, Jasper Lievisse Adriaanse, Theo de Raadt
Subject: Octane status
I am slowly making progress taming the hardware... here is the current
state of HEAD:
- on IP30 (Octane), no improvement, isp(4) eventually gets confused.
- on IP27 (Origin 200) and IP35 (Fuel):
- isp(4) gets better and is able to probe the busses. However there are many
error messages and non-polled I/O never completes.
- this improvement is caused by improvements in DMA operation. As a
result, USB devices now attach properly on Fuel. I have been able to
attach and detach an ukbd, which has been correctly recognized, and
a small umass device, which I could label, newfs, mount and do some
I/O on.
- configuration of PCI devices works, to some extent, and I have been
able to get an fxp(4) recognized in both machines, and able to talk
on the wire (I had to add the fxp firmware on my ramdisk filesystem,
though...)
So this means that, after I buy a large enough USB key tomorrow (the
only umass I have here is a ridiculous 128MB), I'll be able to try to
run multiuser on the Fuel. I'll also try to setup a diskless environment
on the Origin 200, although I'll have to get the fxp firmware compiled
into the kernel for this to work.
There are probably bugs lurking in the IP27 interrupt code, which will
be exposed by this setup. And once they will be fixed, I'll try to
tackle isp(4).
And now, here is the log of a session on the Fuel console, complete with
an openssl benchmark (using a statically linked openssl binary, of
course). Note the openssl results are 3 to 5 times faster than on a
300MHz RM5200 O2.
>> bootp()/bsd.ip27
Setting $netaddr to 10.0.1.212 (from server )
Obtaining bsd.ip27 from server
6106928+485152 entry: 0xa800000000040000
ARCS64 Firmware Version 64.0
Found SGI-IP35, setting up.
config @0x9600000000030000
magic 0xbeedbabe version 0
console 0x920000000f820178 baud 9600
Machine is in M mode.
Region present 0x1.
Calias size 0x2.
board type 11 slot 0 nasid 0 nic 0x553a318d components 4
hub widget 0 port -1 flag 0 speed 200MHz
memory 1024MB, select 0
bank 1 256MB
bank 2 256MB
bank 3 256MB
bank 4 256MB
memory from 0x0 to 0x4000000 (64 MB)
memory from 0x8000000 to 0xc000000 (64 MB)
memory from 0x10000000 to 0x14000000 (64 MB)
memory from 0x18000000 to 0x1c000000 (64 MB)
memory from 0x20000000 to 0x24000000 (64 MB)
memory from 0x28000000 to 0x2c000000 (64 MB)
memory from 0x30000000 to 0x34000000 (64 MB)
memory from 0x38000000 to 0x3c000000 (64 MB)
memory from 0x40000000 to 0x44000000 (64 MB)
memory from 0x48000000 to 0x4c000000 (64 MB)
memory from 0x50000000 to 0x54000000 (64 MB)
memory from 0x58000000 to 0x5c000000 (64 MB)
memory from 0x60000000 to 0x64000000 (64 MB)
memory from 0x68000000 to 0x6c000000 (64 MB)
memory from 0x70000000 to 0x74000000 (64 MB)
memory from 0x78000000 to 0x7c000000 (64 MB)
component widget 0 type 17
cpu type f24/9475 600Mhz cache 4MB speed 300Mhz
board type 42 slot 0 nasid 0 nic 0xffffffffffffffff components 1
xbow hub master link 10
widget 2 nasid 0 flg 6
widget 5 nasid 0 flg 5
widget 6 nasid 0 flg 5
widget 7 nasid 0 flg 5
board type 54 slot d nasid 0 nic 0xffffffffffffffff components 1
component widget 13 type 10
board type 71 slot e nasid 0 nic 0x553a318d components 2
component widget 14 type 5
component widget 14 type 0
board type 71 slot f nasid 0 nic 0x553a318d components 4
component widget 15 type 5
component widget 15 type 37
component widget 15 type 6
component widget 15 type 34
Initial setup done, switching console.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2009 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.5-current (RAMDISK-IP27) #184: Sun May 3 19:32:21 GMT 2009
miod@santoire.gentiane.org:/usr/src/sys/arch/sgi/compile/RAMDISK-IP27
real mem = 1073741824 (1024MB)
rsvd mem = 47194112 (45MB)
avail mem = 1001115648 (954MB)
mainbus0 at root
cpu0 at mainbus0: MIPS R14000 CPU rev 2.4 600 MHz with Unknown FPU type (0x94) rev 7.5
cpu0: cache L1-I 32KB D 32KB 2 way, L2 4096KB 2 way
clock0 at mainbus0: ticker on int5 using count register
xbow0 at mainbus0: XXBow revision 2
"Odyssey" revision 11 at xbow0 widget 13 not configured
xbridge0 at xbow0 widget 14: XBridge revision 2
pci0 at xbridge0 bus 0
fxp0 at pci0 dev 2 function 0 "Intel 8255x" rev 0x0c, i82550: irq 2, address 00:02:b3:98:a0:99
inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
xbridge1 at xbow0 widget 15: XBridge revision 2
pci1 at xbridge1 bus 0
isp0 at pci1 dev 1 function 0 "QLogic ISP12160" rev 0x06: irq 1
isp0: invalid NVRAM header
isp0: invalid NVRAM header
scsibus0 at isp0: 16 targets, initiator 7
isp0: Unhandled Response Type 0x0
isp0: Not RESPONSE in RESPONSE Queue (type 0x0) @ idx 0 (next 1) nlooked 1
sd0 at scsibus0 targ 1 lun 0: <IBM, DDYS-T18350N, SB0A> SCSI3 0/direct fixed
sd0: drive offline
scsibus1 at isp0: 16 targets, initiator 7
ioc0 at pci1 dev 4 function 0 "SGI IOC3" rev 0x01
onewire0 at ioc0
ioc0: superio irq 0, ethernet irq 4
com0 at ioc0 base 0x00020178: ns16550a, 16 byte fifo
com0: console
com1 at ioc0 base 0x00020170: ns16550a, 16 byte fifo
ohci0 at pci1 dev 5 function 0 "Opti 82C861" rev 0x10: irq 5, version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0 "Opti OHCI root hub" rev 1.00/1.00 addr 1
rd0: fixed, 8192 blocks
umass0 at uhub0 port 2 configuration 1 interface 0 "NewTech Inc. USB Mass Storage Device 2.0" rev 2.00/1.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus2 at umass0: 2 targets, initiator 0
sd1 at scsibus2 targ 1 lun 0: <USB, FLASH DISK V2.0, 3002> SCSI2 0/direct removable
sd1: 124MB, 512 bytes/sec, 255744 sec total
boot device: sd0
root on rd0a swap on rd0b dump on rd0b
WARNING: No TOD clock, believing file system.
WARNING: CHECK AND RESET THE DATE!
erase ^?, werase ^W, kill ^U, intr ^C, status ^T
(I)nstall, (U)pgrade or (S)hell? S
# ifconfig fxp0 10.0.1.233
# ifconfig fxp0
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:02:b3:98:a0:99
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 10.0.1.233 netmask 0xff000000 broadcast 10.255.255.255
inet6 fe80::202:b3ff:fe98:a099%fxp0 prefixlen 64 scopeid 0x1
# mount -t nfs -o bg,soft,intr 10.0.1.101:/ftp /mnt2
# disklabel sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label: FLASH DISK V2.0
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 15
total sectors: 255744
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
c: 255744 0 unused
# disklabel -E sd1
Label editor (enter '?' for help at any prompt)
> z
> a
partition: [a]
offset: [0]
size: [255744]
FS type: [4.2BSD]
> w
> q
No label changes.
# newfs -q sd1a
/dev/rsd1a: 124.9MB in 255744 sectors of 512 bytes
4 cylinder groups of 31.22MB, 1998 blocks, 4096 inodes each
# mount /dev/sd1a /mnt
# cd /mnt
# cp /mnt2/pub/OpenBSD/snapshots/sgi/
INSTALL.sgi bsd etc45.tgz misc45.tgz xfont45.tgz
MD5 bsd.rd game45.tgz xbase45.tgz xserv45.tgz
base45.tgz comp45.tgz man45.tgz xetc45.tgz xshare45.tgz
# cp /mnt2/pub/OpenBSD/snapshots/sgi/bsd .
# cd /
# umount /mnt
# mount
/dev/rd0a on / type ffs (local)
10.0.1.101:/ftp on /mnt2 type nfs (v3, udp, soft, intr, timeo=100)
# sd1 detached
scsibus2 detached
umass0 detached
#
# umount /mnt2
# mount -t nfs -o bg,soft,intr 10.0.1.101:/users /mnt
# /mnt/miod/openssl speed -elapsed
You have chosen to measure elapsed time instead of user CPU time.
To get the most accurate results, try to run this
program when this computer is idle.
Doing md2 for 3s on 16 size blocks: 75069 md2's in 3.02s
[...]
Doing 2048 bit verify dsa's for 10s: 357 2048 bit DSA verify in 10.02s
OpenSSL 0.9.8k 25 Mar 2009
built on: date not available
options:bn(64,64) md2(int) rc4(ptr,int) des(ptr,risc2,4,int) aes(partial)
+blowfish(ptr)
compiler: information not available
available timing options: USE_TOD HZ=100 [sysconf value]
timing function used: gettimeofday
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md2 397.77k 862.59k 1206.43k 1340.22k 1384.72k
mdc2 0.00 0.00 0.00 0.00 0.00
md4 3038.31k 10572.88k 30006.93k 56058.23k 75374.34k
md5 2373.23k 8168.48k 22910.45k 41947.58k 55435.35k
hmac(md5) 2881.94k 9541.23k 25468.98k 43950.87k 56015.52k
sha1 2313.53k 7692.07k 19743.66k 32474.73k 40025.63k
rmd160 995.99k 2444.79k 4514.79k 5729.98k 6218.36k
rc4 48197.51k 55976.50k 58301.53k 58929.03k 59099.04k
des cbc 10476.48k 11178.67k 11351.42k 11395.76k 11397.73k
des ede3 3953.03k 4051.04k 4075.34k 4081.35k 4081.17k
idea cbc 0.00 0.00 0.00 0.00 0.00
seed cbc 0.00 0.00 0.00 0.00 0.00
rc2 cbc 6446.43k 6685.48k 6746.95k 6762.23k 6766.22k
rc5-32/12 cbc 0.00 0.00 0.00 0.00 0.00
blowfish cbc 17894.74k 19536.61k 20041.38k 20168.70k 20205.25k
cast cbc 12206.88k 13109.61k 13340.35k 13399.83k 13413.84k
aes-128 cbc 15667.17k 16594.19k 16867.06k 16934.30k 16950.66k
aes-192 cbc 13795.92k 14535.70k 14746.26k 14797.12k 14809.68k
aes-256 cbc 12356.77k 12931.90k 13096.85k 13137.03k 13146.48k
camellia-128 cbc 0.00 0.00 0.00 0.00 0.00
camellia-192 cbc 0.00 0.00 0.00 0.00 0.00
camellia-256 cbc 0.00 0.00 0.00 0.00 0.00
sha256 2133.24k 4972.56k 8839.23k 10993.27k 11832.71k
sha512 1908.64k 7830.54k 12894.19k 18792.53k 21679.81k
aes-128 ige 15971.20k 17650.82k 17838.77k 17841.60k 17993.71k
aes-192 ige 14203.54k 15277.27k 15463.21k 15484.90k 15675.03k
aes-256 ige 12654.50k 13516.71k 13668.84k 13736.89k 13811.78k
sign verify sign/s verify/s
rsa 512 bits 0.003300s 0.000270s 303.1 3699.4
rsa 1024 bits 0.015298s 0.000724s 65.4 1380.5
rsa 2048 bits 0.087327s 0.002358s 11.5 424.1
rsa 4096 bits 0.568740s 0.008400s 1.8 119.1
sign verify sign/s verify/s
dsa 512 bits 0.002735s 0.002964s 365.6 337.4
dsa 1024 bits 0.007250s 0.008567s 137.9 116.7
dsa 2048 bits 0.023327s 0.028059s 42.9 35.6
# umount /mnt
#
While tinkering with interrupts, I stumbled upon a bad bug in the clock interrupt handling code.
Date: Mon, 4 May 2009 17:47:51 +0000 From: Miod Vallat To: Joel Sing, Theo de Raadt, Mark Kettenis Subject: sgi clock fix [...] Basically, the bug is so stupid noone had noticed it. We run with the clock interrupt always enabled, and each time we get a clock interrupt, we count how many integral ticks have elapsed since the last interrupt, schedule the next one, and if we are not at splclock() or splhigh(), invoke hardclock() as many times as necessary. Since the clock is implemented with a free-running counter, we have to make sure, when we schedule the next interrupt, that the counter has not already reached it. You might remember I fixed a similar issue on hppa a few months ago. If we notice we are getting behind, we then schedule the next interrupt one tick later and count one more pending tick. This comparison was not made using signed arithmetic in the original code, so it was always true. The code thus thought it was getting late, and would schedule the next interrupt 1/hz second later. So instead of getting 100 interrupts per second and invoking hardclock on every interrupt, we were getting 50 interrupts per second and invoking hardclock twice on every interrupt. Timekeeping would be ok, but latency would be terrible. [...]
The next day, I made progress on the internal SCSI controller, but the results were frustrating.
Date: Tue, 5 May 2009 22:13:14 +0000 From: Miod Vallat To: Joel Sing Subject: isp I've been concentrating my work on Fuel recently, and since a couple hours I have been able to get isp(4) to correctly see the disks and do I/O. So I was installing from the network, but eventually disk I/O froze while unpacking base45 (but the rest of the machine was alive, the ``stalled'' progress bar was being refreshed, machine would answer pings, etc). I need to investigate further (that's an ISP12160 chip on this machine, BTW). And I also need to clean up my changes, test them on Octane and try to go further.
A few fixes later, I was able to run multiuser on the Origin 200, but disk I/O were extremely slow. The next step, besides figuring out the reason for that terrible slowness, was to be able to boot from disk.
At first, I thought that, just like for the kernels due to different link address on the various SGI families, there would be a need for one bootloader binary per family.
But discussing this with other developers, Mark Kettenis pointed that ARCLoad, the bootloader now used by Linux, was documented to be able to load kernels on IP27, IP28 and IP30 systems from a single binary.
I had a quick look at its code, and realized that the binary was built as a relocatable file, instead of a standalone ELF binary. This way, it did not have to require a specific address where to be loaded in memory, but would let ARCS load it at the address of its choice.
Date: Sat, 9 May 2009 21:04:43 +0000 From: Miod Vallat To: Mark Kettenis Cc: Joel Sing, Theo de Raadt, Kenneth R. Westerback Subject: Re: sgi 64 bit bootblocks, second part > Hmm, http://www.linux-mips.org/wiki/ARCLoad claims the 64-bit version > supports IP27, IP28 and IP30, and looking at the source code, there > doesn't seem to be any model-specific switches in the makefiles beyond > the 32-bit vs. 64-bit switch. And it builds a relocatable file. So this means ARCBios is more smart [sic] than I thought (at least the 64 bit version). If this is true, and if I can get this to work, we'll need only two bootloaders (and the cdrom image will be bootable on all systems).
My next status update showed some progress.
Date: Sun, 10 May 2009 21:19:19 +0000
From: Miod Vallat
To: Joel Sing, Theo de Raadt, Jasper Lievisse Adriaanse
Subject: SGI status
Status update of the weekend.
IP30 / Octane:
still works fine. No network. There is a PCI cardcage on ebay
for USD 160, and the guy will only ship to the US, so it's not
worth looking for.
Known problems:
- root device computation is wrong - it is based on the
simplified O2 logic, while we need to move towards a
device_register() based logic. This is quite simple to do.
IP27 / Origin 200:
no improvement, on-board isp(4) is still slow. I swapped the
drive with one of the Octanes to confirm this is not a disk
problem. I believe some interrupts are stuck at the xbridge
level, but I can't confirm it yet; but I know what code to write
to confirm this. On the other hand that might be a problem
specific to isp(4), since the fxp(4) card I installed doesn't
have any problems.
Known problems:
- reboot or halt do not work - it looks like I need to flush all
tlb and restore ARCBios exception vectors before invoking
ARCBios to return to the prom.
- root device computation is wrong - same as Octane above.
IP35 / Origin 300: (tested by Jasper)
My heuristic to figure out the serial interrupt did not work on
this machine. I need to write brute force code (all pci
interrupts mapped to serial) and have Jasper test so that I know
the correct interrupt pin, then try and figure out why it is
different than on Fuel.
IP35 / Fuel:
I have been able to complete a make build, with network on
fxp(4), and storage on a 4GB SDHC card connected as umass(4),
with /usr/src on NFS and /usr/obj on umass. Build took 26 hours.
Unfortunately, when I wanted to install an ahc(4) pci card to
drive the local disk, I realized this machine has 3.3V PCI slots
and all my ahc cards are 5V.
The on-board isp(4) doesn't show the same lost interrupts issue
as on Origin 200, but after a while I/O return weird results
when reading back from the disk (causing ffs panics or sgivol
pretending the volume header does not exist), and the machine
freezes completely.
Known problems:
- same reboot problem as on Origin 200
- same root device problem as on Origin 200
- onboard isp(4) not reliable.
Boot blocks:
I have not been able to get relocatable binaries loaded by the
PROM. An arcload binary extracted from debian packages works,
the same arcload binary recompiled under OpenBSD fails with
relocation errors, likely due to toolchain differences. So I'll
settle with three non-relocatable bootblocks for now, and I'll
give this another try after I complete the binutils update.
Note that I am inclined to believe the isp(4) problems are
driver-specific, rather than pci controller-specific, as all the other
pci devices appear to work fine (fxp, ohci).
This is because the IOC3 chip, despite being attached to a PCI bus, violates the PCI specification in multiple ways.
Depending on where the IOC3 chip was supposed to be used, it could be built with only a subset of its components: for example, when put on the PCI CADDuo card, it would only contain the PS/2 keyboard and mouse ports, and the Ethernet interface; but when put on the MENET multi-Ethernet card, there would be four IOC3 chips, the first three having only the Ethernet interface and the two serial ports, and the last one having only the Ethernet interface.
Unfortunately, the SGI engineers who designed that chip decided to use two interrupt lines, one for the Ethernet part, and one for all the other functions.
The PCI specification requires that a given PCI device can only use a single interrupt line, unless it advertizes itself as a multi-function PCI device (with up to eight functions), and each function can use a separate interrupt line.
Of course, the IOC3 chip does not report itself as a multi-function device, and there is no way to know that it uses a second interrupt, and which one. And, just to make our lives difficult, that unknown second interrupt, would be the non-Ethernet interrupt, the one needed to make the serial console work.
If the Ethernet part was missing, of course, there would be a single interrupt line and no trouble.
In these early days of IOC3 support, we were focusing on the on-board IOC3 found on all these machines, and since the interrupt assignment for the on-board devices was fixed, I had the impression that it would use the next unused interrupt pin for the second interrupt, because it appeared to work that way on Octane and Origin 200. But this did not work on Fuel and I had to use horrible tricks to detect the proper interrupt line to use.
This was the first and only time I used the word defecate in a commit message.
From: Miod Vallat
Date: Wed, 27 May 2009 19:04:47 +0000
To: openbsd-cvs
Subject: CVS: cvs.openbsd.org: src
CVSROOT: /cvs
Module name: src
Changes by: miod@cvs.openbsd.org 2009/05/27 13:04:47
Modified files:
sys/arch/sgi/pci: ioc.c
sys/arch/sgi/xbow: xbridge.c
Log message:
Yet another attempt at a more reliable detection of the second interrupt
used by onboard IOC chips, by forcing the IOC to trigger this interrupt,
and some help from the PCI bridge driver to report which interrupt has
fired through a fake PCI configuration register.
This works nicely on IP27 and IP35, but on IP30 the interrupt doesn't
happen, for some reason; so keep the existing heuristic in case the above
trick did not give us a valid interrupt number.
In case we got an interrupt, this will also detect IOC configurations where
there is actually one interrupt, should such configurations exist.
<rant style="beck">
I probably deserve to rot in hell for this abomination, but I won't mind
as long as the IOC designers who came with the bright ``let's use more than
one interrupt and defecate on the pci spec'' ideas are there, too.
</rant>
Eventually, after a lot of testing, hair pulling, reading the Linux source code and getting a CADDuo card on eBay, a much better algorithm was devised.
From: Miod Vallat
Date: Wed, 11 Nov 2009 15:29:31 +0000
To: openbsd-cvs
Subject: CVS: cvs.openbsd.org: src
CVSROOT: /cvs
Module name: src
Changes by: miod@cvs.openbsd.org 2009/11/11 08:29:31
Modified files:
sys/arch/sgi/pci: ioc.c
sys/arch/sgi/xbow: xbridge.c
Log message:
It turns out PCI IOC3 card which embed both the Ethernet controller and the
superio chip interrupt on two different pins (yet do not advertize
themselves as a multi-function device, of course).
So, on one hand, this makes the ioc attachment code simpler, because it
simply needs to map interrupt pins A and B, and another hand, this moves
all the interrupt knowledge to the PCI bridge driver, since routing of pin
B differs whether the device is the onboard IOC3 chip (and able to use
any of the 8 bridge interrupt sources...) or on a PCI board (with pin
mapping sane, since controlled by the bridge).
This makes superio interrupts on CADduo boards work. Tested to cause
no regressions on Origin 200, Octane and Fuel.
Despite what I wrote regarding the boot blocks, I suceeded in building relocatable boot blocks for 64-bit ARCS, and they were added to the source tree on may 14th.
The slow disk I/O performance on Origin 200 turned out to be a known SGI chip errata, documented in IRIX headers, for which I had added the advised workaround, but I had made a mistake in my code and it did not actually work around the issue correctly. On may 15th, I fixed my code and suddently the Origin 200 became usable.
Usable, but not stable.
Date: Thu, 21 May 2009 19:47:39 +0000 From: Miod Vallat To: Joel Sing Subject: sgi status [not including deraadt and kettenis as they are AFK] With the interrupt fixes in, things are in better shape, but there's still work to do. * Origin 200 The machine freezes for no apparent reason when compiling stuff or whatnot. I am able to reproduce this with `openssl speed -elapsed' systematically, it always happens at the same time. Even in single user. I need to write support for the NMI button, and then maybe I'll be able to figure out what's wrong. * Octane Still lots of problems with isp. Smells like a magic flag I forgot to set or clear in the bridge registers. This will hopefully work again when I'm done rewriting PCI resource allocation in xbridge. * Fuel All the problems I blamed on isp where actually caused by fxp. It turns out that, right now, I am trying to use the 2GB direct DMA window, and limiting bus_dmamem_alloc() to return memory in this range. However, mbuf are not allocated that way, and bus_dmamap_load_mbuf() - or even bus_dmamap_load() in the case of fxp - will be invoked on arbitrary memory. To fix this, I need to write support for the IOMMU part of the Bridge, and manage its ATE memory. However, this ATE memory is limited (especially on older Bridge vs XBridge). The Bridge can use ATE in external memory, but this memory is limited, and there is a bug which prevents that memory to be modified while there is a DMA transfer occuring, which means we can not really use it. So I'll write some iommu code to cope with situations like that, and upon lack of resources, the stack will try again later when resources have been freed. In the meantime I removed half the memory of the Fuel machine, so that all the physical memory is below the 2GB limit, and everything worked fine. The machine is currently baking a muild on local disk. * Left to do - xbridge pci resource management rewrite - xbridge iommu basics - nmi handling on ip27 - reboot code on ip27 and ip35 - integrate ip27 and ip30 kernels into release builds - more hw support (keyboard/mouse controller, video, ethernet...)
Shortly after this mail was sent, the build on Fuel completed.
<miod> yummy, that 600MHz SGI Fuel bakes a muild with src and obj on local disk
in 6 hours and a half. which is about the same build time as a 400MHz
hppa (B2000).
<jsing> nice! :)
<miod> i think the 4MB L2 cache helps a lot.
(Note the expression "bakes a muild" above - the command used to build a complete OpenBSD userland is "make build", but since it makes systems run hot and can take quite some time, at some point in a discussion in october 2004 I jokingly exchanged letters to coin the "bake muild" expression, and it has stuck to the point of becoming commonly used by other developers since.)
On may 25th, after many experiments trying to figure out why the Origin 200 would eventually freeze under load, I had an epiphany and found the solution.
Years ago, I fixed an R5000 O2 instability by implementing a workaround for a chip bug, which was supposed to be fixed in that particular revision of the die but wasn't (tlbhandler.S 1.16). Being lazy, I did not write a runtime selection of the appropriate TLB handler code, although this was on my list. It turns out that this fix confuses the hell of R10000 processors revision 3 (but not earlier 2.x revisions), to the point of making the Origin 200 here hang so hard it would not even enter the NMI handler (don't ask me how I figured this was the cause). So it's time to choose the appropriate TLB handling flavour at runtime, building the trampoline code from the fixed exception handler location jumping to the handler address at runtime. As a bonus, kernels linked in KSEG0 get the address computation optimized and thus a smaller trampoline than before.
Interestingly, the R5000 errata commit I was referring to had also been done on a may 25th, exactly two years before!
With the Origin 200 runnning stably now, I added the IP27 and IP30 kernel to the OpenBSD/sgi builds on may 29th.
On june 2nd, support for these systems was made public.
Date: Tue, 02 Jun 2009 01:38:59 +0000 From: Miod Vallat To: openbsd-sgi Subject: Fuel, Octane and Origin 200 support Hello folks, I am glad to announce that we have reached a state where support for Fuel, Octane and Origin machines is worth testing. There are still known problems, which are being worked on, but these machines are however usable if you run the latest OpenBSD/sgi snapshot (binaries from May 31th). What works: - Fuel, Octane and Origin 200 boot multiuser. - On-board SCSI controllers are supported. - On-board USB controller on Fuel is supported. - PCI devices in Fuel and Origin 200 are expected to work. I use fxp(4) network cards in both Fuel and Origin 200, and will try more devices soonish. Some cards will not configure correctly because of limitations in my code, which I am working on. - For the Octane owners with PCI card cages, these should be supported as well as the Fuel and Origin 200 PCI slots, but this has not been tested. What doesn't work: - The onboard Ethernet interface on theses machines is not supported yet (which is why I am using fxp PCI network cards). This means that the Octane systems without PCI card cages are pretty useless at the moment. - Neither the PS/2 keyboard and mouse ports nor the graphics options are supported yet; serial console is required at the moment. - On multiprocessor machines, only one processor will be used at the moment. What might work: - Other systems of similar designs will probably work to some extent, but we won't know this until someone having access to these systems tests OpenBSD on them and reports how it went. Such systems are: Origin 2000, Onyx 2, Origin/Onyx 300, Origin/Onyx 3000. If your system has multiple nodes, only the resources of the first node will be used. What will not work: - The Origin/Onyx 350, Onyx 4 and Tezro could run, but there is no support for their PCI-X controller yet (which means no devices will be detected on them, not even the onboard devices). This will not change until a developer can get access to such a machine. Enjoy! Miod PS: I should thank Jean-Marc Harang for lending me a Fuel system to work on; this has helped immensely.
<kettenis> hmm, sgi feels much faster <deraadt> because of? <miod> because of speed! (-: <kettenis> probably the stuff that miod fixed a while ago <miod> interrupts? oh yes, that helped. <miod> getting faster machines to work also helped (-: <kettenis> am I halucianting? <deraadt> now you gotta get the even faster ones working. <deraadt> no kidding. where is the real miod <deraadt> is this reyk playing a joke on miod in return? <beck> holy fuck.. theo you better turn off miod's account. it's been compromised.
Theo de Raadt got a 700MHz R16000 Fuel system on june 16th, to use as a build machine for the OpenBSD/sgi snapshots.
I also bought one (albeit 600MHz only) and received it on june 23rd, which allowed me to return the Fuel I had been using to its owner to complete the dare.
After setting up his machine with an fxp Ethernet interface, Theo de Raadt wanted to put a bge card in order to have a faster (gigabit) link. But the card would not work, and he asked me whether I could fix this.
The reason why this board did not work is that this is a PCI64 card, able to work with 64-bit wide bus addresses, and my code responsible for setting up the PCI bus resources did not handle this particular case, for I had not thought of testing that case. (To my defense, I am not sure I had any such PCI card to test, to begin with.)
Reworking that code turned out to be rabbit's hole, and I ended up eventually writing complete code to set up the resources of a complete PCI bus and the underlying buses, if any (PCI-PCI bridges, or PCI-Cardbus bridges.)
With the 4.6 release cutoff being close, I aimed for small changes, though, and only a subset of these changes were commited on july 6th to make the release, allowing bge cards to work. The remainder of the code, with support for underlying buses, went in one week later, after the 4.6 release had been tagged.
I thought it was time for a well-deserved break.
Date: Tue, 28 Jul 2009 18:35:44 +0000
From: Miod Vallat
To: Theo de Raadt, Jasper Lievisse Adriaanse, Joel Sing
Subject: non-O2 sgi status
I've been doing mostly sgi work in the last four months, and I'll need a
break from it shortly, to work on some MI stuff (wscons, etc) and spend
time on other architectures needing some love.
That's where you guys can step up (-:
Here is a short list of the problems I am aware of in the current
OpenBSD/sgi code for non-O2 machines:
* HARDWARE SUPPORT
The following two points need to be worked on as soon as possible:
- the serial ports do not work on Origin class machines, except for
the console port, while on Octane both serial ports work correctly.
I am currently investigating this, as this could be part of the
reasons why interrupt-driven serial output on Jasper's Origin 300
does not work.
- no onboard Ethernet controller support. This is a pre-MEC design, so
most of the MEC code can be reused, but since there is no per-txdesc
error information, we have to ask it to interrupt after each TX. No
wonder why its performance, even under IRIX, sucks.
The following points need to be worked on eventually, but it's not as
important as the above two at the moment.
- no keyboard/mouse controller support. This is almost trivial to do,
I'll probably give it a try unless Joel beats me to do it.
- video support for Impact-style graphics (found on Octane) and
VPro-style graphics (found on Octane and Fuel). There is code for
both in the linux-mips tree (or maybe with skylark's octane
patches), so it can be done eventually.
- PIC and IOC4 support for Origin 350 and Tezro support. Probably not
worth doing until the Origin 300 problems are fixed, because Origin
350 is likely to have the same. I think I have enough PIC
information, as for IOC4 I think I can support everything on it but
the atapi controller, which will need serious tinkering (but then
only the DVD drive is connected to it, so it's not vital for system
use).
* MI CONCERNS
We are only using memory under 2GB physical so far, because of DMA
concerns. The reason behind this is that the PCI bridge chip (driven
by xbridge(4) ) has a very limited number of IOMMU entries, so we are
falling back on a direct DMA window, which only spans 2GB.
This means that on multiple node Origin 200/2000 systems, we are only
using memory from the first node; and that on Fuel and Origin 300, we
are only using memory from the first bank.
However, devices able to use 64 bit DMA address can do so, and do not
need to go through the direct DMA window.
Ideally, we should be able to solve this in the MI code, by having the
MI code being able to know if the given mbuf or buf is going to go
through a DMA-limited device; and if so, check the DMA constraints and
perform bounce buffer allocation if needed. That's easier said than
done.
In the meantime, I can provide diffs which force mbuf pool allocators
to only use the low memory (but then devices such as the onboard
interface do 64 bit DMA so wouldn't need it, when supported), and to
only allocate physical memory for the buffers from the low memory (but
then, on machines with a lot of memory, we want the buffer cache to be
able to use more than 2GB of memory).
* SMP
The current code assumes it runs on CPU0. I plan to slowly move to
per-cpu_info interrupt information (i.e. interrupt mask and
acknowledge pointers, etc), so that if CPU0 is disabled or missing, we
can run on another processor.
Then I'll tackle the SMP work; however I'll probably want to take some
time to make the interrupt mess^Wcode cleaner (since that impacts the
to-be-written MP-compliant mutex code).
From my point of view, the most important thing we need during this
release cycle is hardware support, however. Being able to support the
on-board Ethernet will bring us many Octane users who can't find a
stable linux distro and are becoming fed up with IRIX, and even more so
if we have some decent glass console capabilities. Most Octane users
won't even notice the 2GB limit because there aren't many Octane systems
with more than 2GB; and they'll cope with the lack of SMP for a while.
Then of course to get fast build machines, it would be nice to make some
progress on Origin 300 and 350. But that's slightly less important.
Miod
PS: Oh, and I still intend to support the lower class IP22 machines
(Indy, Indigo 2) as long as they have processors without too many
hardware bugs in 64 bit mode. But I only have the boot blocks for them
so far (-:
(skylark in the message above being Stanislaw Skowronek.)
On september 1st, while searching for some Octane information, I stumbled upon the blog of Takuya Asada. He had a dual-processor Octane 2 system, had noticed the OpenBSD work, and was experimenting with multiprocessor support. You can e.g. look at his posts during the month of july where most of them are focusing on the Octane.
This was too good to be true, so I contacted him immediately to ask whether he would be interested to work on multiprocessor support with us.
Date: Tue, 1 Sep 2009 21:10:59 +0000 From: Miod Vallat To: Takuya Asada Cc: Jasper Lievisse Adriaanse, Joel Sing, Theo de Raadt Subject: SMP on OpenBSD/sgi Hello, I have just discovered your SMP work for OpenBSD on the Octane (on http://d.hatena.ne.jp/syuu1228/ ). Unfortunately I don't speak Japanese, and the output of the automatic translation tools is sometimes rough to read... If you don't recognize my name, I am one of the OpenBSD developers, and the person who wrote most of the Octane and Origin family support over the last few months. This is why your work has a lot of interest to me (-: I know from experience that adding SMP support to an OpenBSD architecture is a lot of tedious work, and I am really impressed by the work you have done already. Adding SMP to the sgi codebase, for both the Origin and the Octane systems, is on my list of things to do; and I was about to start doing preliminary work for this (writing SMP mutex code, adding locks around interrupts and traps, adding more information to struct cpu_info, etc). Most of this would duplicate work you have already done. In fact, some of the changes you have made are already found in the current OpenBSD source (such as astpending changed from a global variable to a per-process variable). Would you be interested in working more closely with us, and get your work integrated into the OpenBSD source tree? Regards, Miod PS: One important change I have in mind is to put the address of the per-cpu interrupt registers (on Octane, HEART_IMR(cpu) and HEART_ISR) in struct cpu_info, so that the interrupt code does not need to recompute the address every time. It would make sense to put the address of the IPI register there too. This will make the interrupt handling code faster.
Asada agreed with that offer, and was able to start working directly in the OpenBSD source tree a few days later.
An OpenBSD mini hackathon was planned to happen during one week of november in Coimbra, Portugal, and it was decided that I would ship the dual-processor Octane I was using to the hackathon, and Takuya Asada, Joel Sing and I would attend the hackathon to make progress on Octane multiprocessor support. (I would end up working on Loongson hardware support instead during this hackathon, and did not contribute much to their Octane work there.)
In the meantime, a Fuel was not good enough for Theo. And after that mail, he was aiming for a multiprocessor Origin 350 system. Thanks to developer Ryan McBride and Mark Uemura, a couple of such systems were found and bought in Japan, and Theo was supposed to fly back to his lab with them in september.
Unsurprisingly, the OpenBSD kernel did not run out of the box on the Origin 350. A first difference between the Fuel and the Origin 350, both being of the IP35 family, was that the Origin 350 used a newer version of the horrible IOC3 chip. That IOC4 chip was much better but needed a different driver; I had blindly added a tentative driver for that chip in august, but it had never been tested until Theo tried it. After some painful debugging, it turned out my code did not compute the serial port clocks correctly and was programming wrong time divider values, preventing console input and output from working.
Another difference between the Fuel and the Origin 350, was that the Origin 350 sported a more modern version of the XBridge chip, called the PIC. Unlike the XBridge which supports a single PCI bus, the PIC supports two PCI-X busses. The IRIX header files, as well as the IRIX code contributed to Linux during the short time Linux had support for the Altix 350 (the Itanium-based equivalent of the Origin 350), gave me enough information to blindly write support for the PIC at the same time I had written the IOC4 code.
Once the kernel reached the point where we could get its output, on october 7th, the PIC driver would attach, and its PCI subdevices were detected correctly. But the machine would not boot much further - interrupts were apparently not handled at all, and after a complete FIFO of messages (16 characters) was output to the serial console, nothing happened.
Interrupt handling on these SGI systems is implemented in hardware through message-passing, which was being transparent to the operating system; all it needed to do was to write a proper destination address for interrupt messages in a specific XBridge register, which allowed interrupts to either get lost (if you didn't program the register correctly), or sent to a particular Hub chip associated to a particular CPU (or CPU pair, in the IP35 case).
The Brige, then XBridge, then PIC chip, was supposed to have 64-bit registers. But a mistake in Bridge design caused 64-bit accesses to its registers to not work; all accesses had to be done in 32-bit operations; and as actually all registers only had no more than 32 useful bits, it was fine to perform 32-bit writes.
The PIC attempted to fix this. But while the fix restored proper 64-bit paths, it actually broke 32-bit operation. So all PIC register operations had to be done as 64-bit width load or stores, and this was easy to wrap with ``read register'' and ``write register'' function pointers in the OpenBSD driver.
And then I had an illumination. During interrupt handling setup, a 64-bit interrupt message destination address had to be written to a Bridge register. But since the Bridge registers were actually only 32-bit, there were in fact two registers involved to program this address, one for the "lower half" part of the 64-bit value, and one for the "higher half". But the PIC, having working 64-bit registers, worked differently, by having the "lower half" register, now of the proper width, accept the complete 64-bit value, and the "upper half" register ignore writes.
Date: Wed, 7 Oct 2009 15:29:07 +0000 From: Miod Vallat To: Theo de Raadt Subject: try this on O350 Magic interrupt register value was truncated to 32 bits, hence no interrupts from xbridge to cpu. That diff should fix it nicely. Oh, and I think I have found where the RTC is; I'll need to tinker in ddb later tonight. But right now I'm going to take a nap. [...]
My intuition of simply changing the code to write the complete 64-bit address to the "lower half" register, regardless of the chip, made the Origin 350 unstuck on october 10th, and a few hours later, Theo had already completed building an OpenBSD/sgi snapshot and would use that machine as its build machine from then on.
I could brag about announce Origin 350 support.
Date: Sun, 11 Oct 2009 20:22:38 +0000
From: Miod Vallat
To: sgi@openbsd.org
Subject: Origin 350 support
Hello folks,
I am glad to announce that, after a week of painful remote debugging,
OpenBSD/sgi now supports Origin 350 systems. In fact, the OpenBSD/sgi
snapshots are now built on an Origin 350, since a couple of days.
What prevented Origin 350 systems from working earlier was the lack of
support for their PCI-X bridges, as well as their base I/O board (IO8 or
IO9 boards). Both of these are now supported, although the IO8/IO9 ATAPI
interface, to which the DVD-ROM drive is connected, is not supported
yet.
This means similar systems, such as Onyx 350, Onyx 4 and Tezro should
work as well. if you are the lucky owner of such a system, please give
the latest OpenBSD/sgi snapshot a try, and let us know how it fares.
Enjoy!
Miod
PS: here is an Origin 350 dmesg for your enjoyment... note the processor
has 16MB of cache! That's gorgeous!
[ using 440792 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2009 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.6-current (GENERIC-IP27) #112: Sun Oct 11 12:46:15 MDT 2009
deraadt@sgi.openbsd.org:/sys/arch/sgi/compile/GENERIC-IP27
real mem = 2147483648 (2048MB)
rsvd mem = 47194112 (45MB)
avail mem = 1999663104 (1907MB)
mainbus0 at root
cpu0 at mainbus0 nasid 0: R16000 CPU rev 3.0 1000 MHz, R16000 FPU rev 10.11
cpu0: cache L1-I 32KB D 32KB 2 way, L2 16384KB 2 way
clock0 at mainbus0 nasid 0: ticker on int5 using count register
xbow0 at mainbus0 nasid 0: Bedrock revision 3
xbridge0 at xbow0 widget 15: PIC revision 3
pci0 at xbridge0 bus 0
iof0 at pci0 dev 0 function 0 "SGI IOC4" rev 0x53: irq 0
com0 at iof0 base 0x380: ns16550a, 16 byte fifo
com0: console
com1 at iof0 base 0x388: ns16550a, 16 byte fifo
com2 at iof0 base 0x390: ns16550a, 16 byte fifo
com3 at iof0 base 0x398: ns16550a, 16 byte fifo
iockbc at iof0 base 0x200 not configured
dsrtc0 at iof0 base 0x80000: DS1742W
isp0 at pci0 dev 2 function 0 "QLogic ISP12160" rev 0x06: irq 2
isp0: Board Type 12160, Chip Revision 0x6, loaded F/W Revision 10.4.41
scsibus0 at isp0: 16 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <SGI, ST3146707LC, C730> SCSI3 0/direct fixed
sd0: 140014MB, 512 bytes/sec, 286749488 sec total
sd1 at scsibus0 targ 2 lun 0: <SGI, ST3146707LC, C730> SCSI3 0/direct fixed
sd1: 140014MB, 512 bytes/sec, 286749488 sec total
scsibus1 at isp0: 16 targets, initiator 0
bge0 at pci0 dev 3 function 0 "Broadcom BCM5701" rev 0x15, BCM5701 B5 (0x105): irq 3, address 08:00:69:11:e6:30
brgphy0 at bge0 phy 1: BCM5701 10/100/1000baseT PHY, rev. 0
pci1 at xbridge0 bus 0
bge1 at pci1 dev 0 function 0 "Broadcom BCM5704C" rev 0x03, BCM5704 A3 (0x2003): irq 0, address 00:e0:ed:07:b8:44
brgphy1 at bge1 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge2 at pci1 dev 0 function 1 "Broadcom BCM5704C" rev 0x03, BCM5704 A3 (0x2003): irq 4, address 00:e0:ed:07:b8:45
brgphy2 at bge2 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge3 at pci1 dev 1 function 0 "Broadcom BCM5704C" rev 0x03, BCM5704 A3 (0x2003): irq 1, address 00:e0:ed:07:b8:39
brgphy3 at bge3 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge4 at pci1 dev 1 function 1 "Broadcom BCM5704C" rev 0x03, BCM5704 A3 (0x2003): irq 5, address 00:e0:ed:07:b8:3a
brgphy4 at bge4 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
boot device: sd0
root on sd0a swap on sd0b dump on sd0b
At this point, another OpenBSD developer who only had an Indy system, asked me whether these systems could be supported. I told him this was on my list, but that I would not work on it in the near future. He then asked for advice, in case he would try his luck. I think my answer was interesting enough to be worth sharing here.
Date: Fri, 23 Oct 2009 04:52:34 +0000
From: Miod Vallat
To: other developer
Subject: Indy
Ok, here is a (not short enough) list of what you should know about the Indy.
* IP22/IP24
The Indy is actually IP24, but uses a similar logic board to the Indigo 2
(IP22), and actually reports itself via ARCBios as IP22. You'll eventually need
to tell IP22 and IP24 apart, since (for example) IP24 has no EISA slots.
Also there were flavors of these machines without frame buffers, intended to
be used as servers: the counterpart of the Indy is the Challenge S, while
the counterpart of the Indigo 2 is the Challenge M (Challenge L and XL are
different beasts).
There is code in NetBSD to figure out which particular flavour we are running
on. From now on, I'll use `IP22' to mean `any flavour' unless a particular
flavour is mentioned.
* Firmware
IP22 systems use a 32 bit ARCBios, which can only load ECOFF binaries. Since
our kernel is ELF, we need to provide ECOFF bootblocks.
Our bootblocks currently only support booting from disks. I plan on adding
the ability to load a kernel from the network, so that you could netboot the
bootblocks which would in turn netboot an ELF kernel.
However the ELF->ECOFF conversion of the bootblock image triggers an internal
bug in binutils; so far I am using a GPL ELF->ECOFF tool from the Linux arcload
(their sgi bootloader) sources, and I really ought to get that binutils
upgrade done ASAP so that hopefully I won't need that tool.
* Memory layout
On IP22, physical memory does not start at address zero. There's a 128MB gap
at the bottom of memory. From memory that's an area where the frame buffer
memory resides (but doesn't use the whole 128MB of course).
So this means the kernel needs to be linked at an address slightly after
128MB physical. You'll need to provide a new kernel, GENERIC-IP22, with
something like:
makeoption LINK_ADDRESS="0xffffffff88100000"
to cause it to be linked in KSEG0 after 128MB.
* Hardware components
NetBSD has support for all the Indy devices, except maybe for the onboard ISDN
controller (which we probably won't want to support anyway) and I think one of
the three Indy frame buffer options is not supported. Oh, and the Indycam
too.
IP24 has two GIO slots which allow for some expansion cards to be used (GIO
slots are also found on IP20, but not on IP22). Support for this can be
ported later.
The important devices are:
- the onboard device glue and interrupt logic (imc + hpc + ioc)
-- note that I didn't remember there was an ioc device on IP22 when I
wrote the IOC3 driver (a completely different chip) as ioc(4). For
IOC4 I named it as `IO Four' thus iof(4), so IOC should probably
become `IO One' thus ioo(4).
- the serial ports are driven by the usual Zilog 8530 chip. That's zs(4) on
many of our platforms, unfortunately they did not all track NetBSD up the
same point, so we have a bunch of different, but close to each other,
zs drivers on an MD basis, as well as a much older and buggier MI zs driver
in sys/dev/ic. I intend to clean this mess and only have one updated MI zs
driver, but have never found time to do it properly. Now would be the time...
- the keyboard and mouse ports are also handled by a Z8530. The devices
themselves are regular PS/2 devices.
- on-board Ethernet is based on a Seeq chip. I think a variant of this
chip was used on mac68k, but without DMA. It is best to port the NetBSD
driver, but (of course!) the driver won't port directly, as we use a
different structure to store the Ethernet address and generally do arp
related operations.
- on-board SCSI is the good old WD33C93 chip. This chip was popular at some
point in the workstation market, but did not get that widespread. We have
a very old driver for it for mvme68k, but it is associated with a very
special DMA controller so can't really be reused. On the NetBSD side,
they now have made the driver MI since it is used on a few platforms,
but there have been some issues so only their sgimips port uses the MI
driver yet. Some particular IBM disks confuse the driver when negotiating
the transfer characteristic (speed and sync vs async operation). I'd
advise starting from the MI driver nevertheless, and `update' it from
the NetBSD `scsipi' framework to our `scsi' framework. To help do this,
you should look at the NetBSD cvs history and see what kind of changes
were made when the thorpej-scsipi branch was merged and undo them (look
at a MD wd33c driver, though, as the MI driver has always been written
for scsipi. sys/arch/amiga/dev/sbic.c might be a good source of
inspiration).
So that's a lot of ungrateful tasks to do before you can have a kernel that
works. You should probably start with the serial stuff (just put yet another
copy of the netbsd mi code as md code in the port and wait for the janitor
to clean this...) and the onboard logic stuff. That would get you a kernel
able to print a few things on the console without needing the firmware for
this, and then dying because of the lack of root devices; and then you can
add ethernet and scsi to make this a bit more useful.
I'll clean up my bootblocks diff and send them to you this evening (actually,
I'm probably going to commit a few of them to make things even simpler).
(All this work, including driver cleanup, would eventually happen, a few years later.)
After working on more changes to the interrupt code in order to better support the various interrupt controllers found on the high-end systems, I came back to the onboard Ethernet driver.
Date: Fri, 30 Oct 2009 19:16:29 +0000 From: Miod Vallat To: Jasper Lievisse Adriaanse, Joel Sing Subject: octane onboard ethernet progress X-Status: A I have resumed working on $SUBJECT. I am making progress - I now have the TX path working correctly (at least for short packets). But I can't receive anything yet. I'm currently hammering the RX code, and hopefully I'll have something sort-of-working soon.
And the next day:
Date: Sat, 31 Oct 2009 00:51:49 +0000 From: Miod Vallat To: Joel Sing Cc: Jasper Lievisse Adriaanse Subject: Re: octane onboard ethernet progress [...] > Well, that is still much further than I managed to get - care to share a diff? Of course. Do not mind debug code and bugs in there. My test setup so far consists of booting GENERIC-IP30 single user over the network, and then I have a machine which is not the bootp/tftp server, which runs ``tcpdump -x ether host <ethernet address>''. This machine also has a static arp entry for the Octane, which allows me to send ping for as long as I want without getting `host is down' errors due to the lack of arp answers. On the Octane, when I ifconfig iec0 with an address, I see on the tcpdump machine the arp request followed by an icmp6 sol request. On the tcpdump machine, pinging the Octane shows rx interrupts occuring on the Octane, but with rx descriptor status being invalid, so all packets received are eaten as errors and nothing serious happens. > I have a large chunk of code that should work, minus the fact that I could > never get interrupts. We might be able to recycle or merge some code... The trick to get RX interrupts, is to enable the RX timer interrupt, with a zero timer. This means that we will get interrupts on packet received + timer expiration. With a timer of zero we get the interrupt immediately upon receive. You will probably curse me for choosing different symbolic names for the registers and their bits than you did (and they are also different from my earlier attempts). Miod PS: there is nothing preventing that code to work on GENERIC-IP27 kernels. However if you try it on an IP35 machine (Fuel, Origin 3*), the driver will not know its ethernet address at the moment. I know what to do to get the address, but this will be horrible (I will need to send commands over the internal L1 serial port access to get EEPROM contents and parse them), so I won't do this until the driver is working. [...]
After a few days of work, the driver was in working state and put in the tree. But the RX performance was horrible, and I could not figure out why.
My code also had a bug causing every one out of 64 packets to be lost, and took some time to figure out and fix.
I gave more details about it on the OpenBSD developers chat years later, which is a good thing, because I no longer remember that particular bug!
<Nick> here's an odd one: anyone seen a problem between a BBB and yp? if I run
yp on one of my BBB's, it loses network in a few days. no yp, no network
loss.
<miod> Nick: that reminds me an evil sq(4/sgi) issue which was preventing packets
from a certain size (which yp is fond of but noone else apparently is) to
not get xmit. you might want to try all ping -s values and see if all
sizes are working.
<Nick> sounds like a fun one, I'll do that.
<dlg> around 60 byte ethernet frames might be a toucy spot
<dlg> cos it has to pad the packet on ethernets behalf
<miod> or this could be the iec(4/sgi) bug: every 64th packet get lost. this one
was tough to fix
<dlg> miod: how big was the ring?
<miod> 256 IIRC
<miod> larger than 64 in any case.
<dlg> what was it?
<miod> i botched programming the interrupt enable, so when it would hit the low
threshold it would fail to interrupt for the given descriptor.
<dlg> miod: nice
On november 3rd, I eventually realized I had to align the RX descriptors to 4KB boundaries to get the proper RX performance.
This made the Octane much more useful, and came at the right time, with the november hackathon approaching.
A few days later, on november 8, I had working code to query the L1 interface on IP35 systems to get the onboard Ethernet address, and I could use the onboard interface of my Fuel.
<miod> wow, make build on sgi with 16KB pages takes 15% less time.
This speedup was unexpected (I was expecting about 3%) and convinced me to try and use 16KB pages on OpenBSD/sgi too.
Date: Wed, 2 Dec 2009 06:11:33 +0000 From: Miod Vallat To: private OpenBSD mailinglist Subject: mips64: support for 16KB page size The following diff adds support for kernels with 16KB page size, but does not change the page size yet, so this diff should be harmless at the moment except for possible bugs in it (I'm using it with both 4K and 16K page kernels at the moment). Details: - <machine/param.h> is now required to provide the page size in bits (PAGE_SHIFT). For now it remains 12, but you can play with making it 14 on sgi. - pmap and pte related defines moved away from <mips64/param.h> - pmap structures which used to be 4KB and allocated as pages remain 4KB but are now allocated from a pool, which uses the previous page allocation functions as its own pool allocator. This causes a bit of overhead on 4KB page size kernels, but avoids wasting memory on 16KB page size kernels. - with 16KB pages, UPAGES is only 1, so instead of reserving a TLB pair and aligning the U area on an even TLB boundary, no TLB are reserved and we use a directly translated address which won't fault, for the U area (see vm_machdep.c, context.S) - tlb handling code will now restore the TLB entry mask when necessary, which is needed to handle > 4KB page sizes. Also check for tlbp instruction result correctly and do not consider tlb entry #0 special in any way. - no virtual cache aliasing occur with 16KB page sizes on most machines, so check for this in pmap_prefer(). [...] At any time, you can check what page size your kernel uses with $ sysctl hw.pagesize Miod
While I was tinkering with page sizes, Takuya Asada was slowly making progress on SMP.
<syuu> SMP on SGI worked, a few seconds ;) <oga> until it hit the scheduler? ;) <dlg> it collapsed into a black hole under the weight of its own awesomeness <oga> that machine still aren't dead.
We were still far away from IP27 and IP35 SMP, and Theo would remind me on a regular basis.
<deraadt> my sgi feels faster
<deraadt> but still only about half the speed it should
<miod> boo hoo
<miod> that line is getting old.
<krw> I think if miod keeps halving the distance between current performance and
the maximum, Zeno has some thoughts on when the maximum will be achieved
<miod> bwahahahahaha
<miod> you just made my evening
<krw> :-)
| SGI model | common name | Linux | NetBSD | OpenBSD |
| IP6, IP10 | Personal Iris 4D/2x | complete distribution no X server |
||
| IP12 | Indigo (R3000) | complete distribution | ||
| IP20 | Indigo (R4000) | complete distribution | ||
| IP22 | Indigo2 | complete distribution XL (newport) graphics only |
complete distribution XL (newport) graphics only |
|
| IP24 | Indy | complete distribution XL (newport) graphics only |
complete distribution XL (newport) graphics only |
|
| IP27 | Origin 200, Origin 2000 | complete distribution | complete distribution no SMP |
|
| IP28 | POWER Indigo2 R10000 | same as IP22 | ||
| IP30 | Octane | not-yet-integrated kernel patches X server on Impact only |
complete distribution no graphics no SMP |
|
| IP32 | O2 | complete distribution no R10000 support |
complete distribution | complete distribution |
| IP35 | Fuel, Origin 300, Origin 350, Origin 3000, Onyx 350, Onyx 4, Tezro | complete distribution no graphics no SMP Origin 300 and 3000 not supported |
(Follow this link to go forward to the next part.)