iomapping a big endian area

iomapping a big endian area

[Matthew Wilcox]

On Sat, Apr 02, 2005 at 02:52:14PM -0600, James Bottomley wrote:
> This driver has had it's own different infrastructure for doing this for
> ages, but it's time it used the common one.

Thanks.  I'd been looking at this for a while but hadn't got round tuit yet.

>  #ifdef CONFIG_53C700_LE_ON_BE
>  #define bE	(hostdata->force_le_on_be ? 0 : 3)
>  #define	bSWAP	(hostdata->force_le_on_be)
> +/* This is terrible, but there's no raw version of ioread32.  That means
> + * that on a be board we swap twice (once in ioread32 and once again to 
> + * get the value correct) */
> +#define bS_to_io(x)	((hostdata->force_le_on_be) ? (x) : cpu_to_le32(x))

I raised this with Linus back when he did the original iomap() stuff.
Unfortunately, I think he ignored the question ;-)

My thought on this is that we should encode the endianness of the
registers in the ioremap cookie.  Some architectures (sparc, I think?) can
do this in their PTEs.  The rest of us can do it in our ioread/writeN
methods.  I've planned for this in the parisc iomap implementation but
not actually implemented it.

It doesn't look too hard so I'll commit something to the parisc tree
later that'll let you iomap a BE area.  Do we have any cards that need to be
accessed in a BE way on a LE machine?  (ie x86)

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: iomapping a big endian area

[David S. Miller]

On Sun, 3 Apr 2005 02:37:57 +0100
Matthew Wilcox <matthew@wil.cx> wrote:

> My thought on this is that we should encode the endianness of the
> registers in the ioremap cookie.  Some architectures (sparc, I think?) can
> do this in their PTEs.  The rest of us can do it in our ioread/writeN
> methods.  I've planned for this in the parisc iomap implementation but
> not actually implemented it.

SPARC64 can do it in the PTEs, but we just use raw physical
addresses in our I/O accessors, and in those load/store instructions
we can specify the endianness.

Be careful though.  Endianness can be dealt with on a hardware
level.  Consider a byte access to a 32-bit word sized config space
datum, the PCI controller on a big-endian system will likely byte-twist
the data lanes in order for this to work properly.

It's a subtle issue, and it's explained pretty well in some of the
UltraSPARC PCI controller docs at:

	http://www.sun.com/processors/documentation.html

In particular, "U2P UPA to PCI User's Manual", chapter 10
"Little-Endian Support", has some informative diagrams.

Re: iomapping a big endian area

[Matthew Wilcox]

On Sat, Apr 02, 2005 at 06:38:05PM -0800, David S. Miller wrote:
> > My thought on this is that we should encode the endianness of the
> > registers in the ioremap cookie.  Some architectures (sparc, I think?) can
> > do this in their PTEs.  The rest of us can do it in our ioread/writeN
> > methods.  I've planned for this in the parisc iomap implementation but
> > not actually implemented it.
> 
> SPARC64 can do it in the PTEs, but we just use raw physical
> addresses in our I/O accessors, and in those load/store instructions
> we can specify the endianness.

Ah right.  So you'd prefer an ioread8be() interface?

> Be careful though.  Endianness can be dealt with on a hardware
> level.  Consider a byte access to a 32-bit word sized config space
> datum, the PCI controller on a big-endian system will likely byte-twist
> the data lanes in order for this to work properly.

Yup, PA-RISC PCI adapters (both Dino and Elroy) do the same thing.
The 53c700 driver handles this lack of skewing by xoring the address with 3.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: iomapping a big endian area

[James Bottomley]

On Sun, 2005-04-03 at 04:10 +0100, Matthew Wilcox wrote:
> > SPARC64 can do it in the PTEs, but we just use raw physical
> > addresses in our I/O accessors, and in those load/store instructions
> > we can specify the endianness.
> 
> Ah right.  So you'd prefer an ioread8be() interface?

Actually, ioread8be is unnecessary, but I was planning to add
ioread16/ioread32 and iowritexx be on be variants (equivalent to
_raw_readw et al.)

After all, the driver must know the card is BE, so the routines that
make use of the feature are easily coded into the card, so there's no
real need to add it to the iomem cookie.

Did anyone have a preference for the API?  I was thinking
ioread32_native, but ioread32be is fine too.

James

Re: iomapping a big endian area

[David S. Miller]

On Sat, 02 Apr 2005 21:40:39 -0600
James Bottomley <James.Bottomley@SteelEye.com> wrote:

> After all, the driver must know the card is BE, so the routines that
> make use of the feature are easily coded into the card, so there's no
> real need to add it to the iomem cookie.

Yes, I don't believe it needs to be in the cookie either.

> Did anyone have a preference for the API?  I was thinking
> ioread32_native, but ioread32be is fine too.

I think doing foo{be,le}{8,16,32}() would be consistent with
our byteorder.h interface names.

Re: iomapping a big endian area

[James Bottomley]

On Sat, 2005-04-02 at 20:08 -0800, David S. Miller wrote:
> > Did anyone have a preference for the API?  I was thinking
> > ioread32_native, but ioread32be is fine too.
> 
> I think doing foo{be,le}{8,16,32}() would be consistent with
> our byteorder.h interface names.

Thinking about this some more, I know of no case of a BE bus connected
to a LE system, nor do I think anyone would ever create such a beast, so
our only missing interface is for a BE bus on a BE system.

Thus, I think io{read,write}{16,32}_native are better interfaces ...
they basically mean pass memory operations without byte swaps, so
they're well defined on both BE and LE systems and correspond exactly to
our existing _raw_{read,write}{w,l} calls (principle of least surprise).

James

Re: iomapping a big endian area

[Benjamin Herrenschmidt]

On Sat, 2005-04-02 at 22:27 -0600, James Bottomley wrote:
> On Sat, 2005-04-02 at 20:08 -0800, David S. Miller wrote:
> > > Did anyone have a preference for the API?  I was thinking
> > > ioread32_native, but ioread32be is fine too.
> > 
> > I think doing foo{be,le}{8,16,32}() would be consistent with
> > our byteorder.h interface names.
> 
> Thinking about this some more, I know of no case of a BE bus connected
> to a LE system, nor do I think anyone would ever create such a beast, so
> our only missing interface is for a BE bus on a BE system.

It's more a matter of the device than the bus imho... 

> Thus, I think io{read,write}{16,32}_native are better interfaces ...

I disagree. The driver will never "know" ...

> they basically mean pass memory operations without byte swaps, so
> they're well defined on both BE and LE systems and correspond exactly to
> our existing _raw_{read,write}{w,l} calls (principle of least surprise).

I don't think it's sane. You know that your device is BE or LE and use
the appropriate interface. "native" doesn't make sense to me in this
context.

Ben.

Re: iomapping a big endian area

[James Bottomley]

On Mon, 2005-04-04 at 17:50 +1000, Benjamin Herrenschmidt wrote:
> I disagree. The driver will never "know" ...

? the driver has to know.  Look at the 53c700 to see exactly how awful
it is.  This beast has byte and word registers.  When used BE, all the
byte registers alter their position (to both inb and readb).

> I don't think it's sane. You know that your device is BE or LE and use
> the appropriate interface. "native" doesn't make sense to me in this
> context.

Well ... it's like this. Native means "pass through without swapping"
and has an easy implementation on both BE and LE platforms.  Logically
io{read,write}{16,32}be would have to do byte swaps on LE platforms.
Being lazy, I'm opposed to doing the work if there's no actual use for
it, so can you provide an example of a BE bus (or device) used on a LE
platform that would actually benefit from this abstraction?

James

Re: iomapping a big endian area

[Christoph Hellwig]

On Mon, Apr 04, 2005 at 08:59:03AM -0500, James Bottomley wrote:
> Well ... it's like this. Native means "pass through without swapping"
> and has an easy implementation on both BE and LE platforms.  Logically
> io{read,write}{16,32}be would have to do byte swaps on LE platforms.
> Being lazy, I'm opposed to doing the work if there's no actual use for
> it, so can you provide an example of a BE bus (or device) used on a LE
> platform that would actually benefit from this abstraction?

The IOC4 device that provides IDE, serial ports and external interrupts
on Altix systems has a big endian register layour, and the PCI-X bridge
in those Altix systems can do the swapping if a special bit is set.

In older kernels that bit was set from the driver through a special API,
but it seems the firmware does that automatically now.

Re: iomapping a big endian area

[James Bottomley]

On Mon, 2005-04-04 at 15:16 +0100, Christoph Hellwig wrote:
> The IOC4 device that provides IDE, serial ports and external interrupts
> on Altix systems has a big endian register layour, and the PCI-X bridge
> in those Altix systems can do the swapping if a special bit is set.
> 
> In older kernels that bit was set from the driver through a special API,
> but it seems the firmware does that automatically now.

We already have some unusual code in drivers to support other altix
design ... features ... do you regard this as something that's likely to
be replicated on other platforms, or is it more in the category of a one
off mistake that can be corrected in firmware?

James

Re: iomapping a big endian area

[Jesse Barnes]

On Monday, April 4, 2005 7:25 am, James Bottomley wrote:
> On Mon, 2005-04-04 at 15:16 +0100, Christoph Hellwig wrote:
> > The IOC4 device that provides IDE, serial ports and external interrupts
> > on Altix systems has a big endian register layour, and the PCI-X bridge
> > in those Altix systems can do the swapping if a special bit is set.
> >
> > In older kernels that bit was set from the driver through a special API,
> > but it seems the firmware does that automatically now.
>
> We already have some unusual code in drivers to support other altix
> design ... features ... do you regard this as something that's likely to
> be replicated on other platforms, or is it more in the category of a one
> off mistake that can be corrected in firmware?

The whole line of altix pci bridges can do byteswapping, so it's more than 
just one product that could benefit (however slightly).  Not sure about other 
bridges though.

Jesse

Re: iomapping a big endian area

[Randy.Dunlap]

James Bottomley wrote:
> On Mon, 2005-04-04 at 17:50 +1000, Benjamin Herrenschmidt wrote:
> 
>>I disagree. The driver will never "know" ...
> 
> 
> ? the driver has to know.  Look at the 53c700 to see exactly how awful
> it is.  This beast has byte and word registers.  When used BE, all the
> byte registers alter their position (to both inb and readb).
> 
> 
>>I don't think it's sane. You know that your device is BE or LE and use
>>the appropriate interface. "native" doesn't make sense to me in this
>>context.
> 
> 
> Well ... it's like this. Native means "pass through without swapping"
> and has an easy implementation on both BE and LE platforms.  Logically
> io{read,write}{16,32}be would have to do byte swaps on LE platforms.
> Being lazy, I'm opposed to doing the work if there's no actual use for
> it, so can you provide an example of a BE bus (or device) used on a LE
> platform that would actually benefit from this abstraction?

I would probably spell "native" as "noswap".
"native" just doesn't convey enough specific meaning...

-- 
~Randy

Re: iomapping a big endian area

[Benjamin Herrenschmidt]

> > Well ... it's like this. Native means "pass through without swapping"
> > and has an easy implementation on both BE and LE platforms.  Logically
> > io{read,write}{16,32}be would have to do byte swaps on LE platforms.
> > Being lazy, I'm opposed to doing the work if there's no actual use for
> > it, so can you provide an example of a BE bus (or device) used on a LE
> > platform that would actually benefit from this abstraction?
> 
> I would probably spell "native" as "noswap".
> "native" just doesn't convey enough specific meaning...

But that implies that the driver has to know that the bus and the device
and the CPU are on the same byte endian etc.... that is rather specific,
and if they all know, then they can also just use the correct "be" or
"le" ... I really see no point in "native" abstraction.

Ben.

Re: iomapping a big endian area

[David Vrabel]

James Bottomley wrote:
> so can you provide an example of a BE bus (or device) used on a LE
> platform that would actually benefit from this abstraction?

The Network Processing Engines in the Intel IXP425 are big-endian and
its XScale core may be run in little-endian mode. There's a bunch of
gotchas related to running in little-endian mode so you typically run
the IXP425 in big-endian mode, though.

David Vrabel
-- 
David Vrabel, Design Engineer

Arcom, Clifton Road           Tel: +44 (0)1223 411200 ext. 3233
Cambridge CB1 7EA, UK         Web: http://www.arcom.com/

Re: iomapping a big endian area

[Richard B. Johnson]

On Mon, 4 Apr 2005, David Vrabel wrote:

> James Bottomley wrote:
>> so can you provide an example of a BE bus (or device) used on a LE
>> platform that would actually benefit from this abstraction?
>
> The Network Processing Engines in the Intel IXP425 are big-endian and
> its XScale core may be run in little-endian mode. There's a bunch of
> gotchas related to running in little-endian mode so you typically run
> the IXP425 in big-endian mode, though.
>

But the Linux interface (on the CPU side of the PCI bus interface)
doesn't care about the implimentation details in the XScale
Core. That's why it's a complete subsystem, isolated from the
ix86 by the PCI/Bus interface.

> David Vrabel
> -- 
> David Vrabel, Design Engineer
>
> Arcom, Clifton Road           Tel: +44 (0)1223 411200 ext. 3233
> Cambridge CB1 7EA, UK         Web: http://www.arcom.com/


Cheers,
Dick Johnson
Penguin : Linux version 2.6.11 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by Dictator Bush.
                  98.36% of all statistics are fiction.

Re: iomapping a big endian area

[David Vrabel]

Richard B. Johnson wrote:
> 
> But the Linux interface (on the CPU side of the PCI bus interface)
> doesn't care about the implimentation details in the XScale
> Core. That's why it's a complete subsystem, isolated from the
> ix86 by the PCI/Bus interface.

Hmmm.

*takes a long hard look at the IXP425 based board in front of him*

You're right.  It's really is a PCI add-on card for an x86 platform.
All this time I've been thinking it was a standalone processor board.
Thanks for clearing that up!

David Vrabel

ps. The IXP425 is an ARM (Intel's XScale architecture) based processor.

Re: iomapping a big endian area

[James Bottomley]

On Mon, 2005-04-04 at 16:41 +0100, David Vrabel wrote:
> The Network Processing Engines in the Intel IXP425 are big-endian and
> its XScale core may be run in little-endian mode. There's a bunch of
> gotchas related to running in little-endian mode so you typically run
> the IXP425 in big-endian mode, though.

Yes, based on feedback from Mips people and others pointing out the
existence of the motorola rapidio bus, which is BE, I give in and agree
that the io{read,write}{16,32}be are the way to go.

How does the attached look?

James

===== arch/parisc/lib/iomap.c 1.1 vs edited =====
--- 1.1/arch/parisc/lib/iomap.c	2005-01-10 08:00:46 -06:00
+++ edited/arch/parisc/lib/iomap.c	2005-04-04 13:53:49 -05:00
@@ -43,10 +43,14 @@
 struct iomap_ops {
 	unsigned int (*read8)(void __iomem *);
 	unsigned int (*read16)(void __iomem *);
+	unsigned int (*read16be)(void __iomem *);
 	unsigned int (*read32)(void __iomem *);
+	unsigned int (*read32be)(void __iomem *);
 	void (*write8)(u8, void __iomem *);
 	void (*write16)(u16, void __iomem *);
+	void (*write16be)(u16, void __iomem *);
 	void (*write32)(u32, void __iomem *);
+	void (*write32be)(u32, void __iomem *);
 	void (*read8r)(void __iomem *, void *, unsigned long);
 	void (*read16r)(void __iomem *, void *, unsigned long);
 	void (*read32r)(void __iomem *, void *, unsigned long);
@@ -122,9 +126,13 @@
 static const struct iomap_ops ioport_ops = {
 	ioport_read8,
 	ioport_read16,
+	ioport_read16,
+	ioport_read32,
 	ioport_read32,
 	ioport_write8,
 	ioport_write16,
+	ioport_write16,
+	ioport_write32,
 	ioport_write32,
 	ioport_read8r,
 	ioport_read16r,
@@ -146,11 +154,21 @@
 	return readw(addr);
 }
 
+static unsigned int iomem_read16be(void __iomem *addr)
+{
+	return __raw_readw(addr);
+}
+
 static unsigned int iomem_read32(void __iomem *addr)
 {
 	return readl(addr);
 }
 
+static unsigned int iomem_read32be(void __iomem *addr)
+{
+	return __raw_readl(addr);
+}
+
 static void iomem_write8(u8 datum, void __iomem *addr)
 {
 	writeb(datum, addr);
@@ -161,11 +179,21 @@
 	writew(datum, addr);
 }
 
+static void iomem_write16be(u16 datum, void __iomem *addr)
+{
+	__raw_writew(datum, addr);
+}
+
 static void iomem_write32(u32 datum, void __iomem *addr)
 {
 	writel(datum, addr);
 }
 
+static void iomem_write32be(u32 datum, void __iomem *addr)
+{
+	__raw_writel(datum, addr);
+}
+
 static void iomem_read8r(void __iomem *addr, void *dst, unsigned long count)
 {
 	while (count--) {
@@ -217,10 +245,14 @@
 static const struct iomap_ops iomem_ops = {
 	iomem_read8,
 	iomem_read16,
+	iomem_read16be,
 	iomem_read32,
+	iomem_read32be,
 	iomem_write8,
 	iomem_write16,
+	iomem_write16be,
 	iomem_write32,
+	iomem_write32be,
 	iomem_read8r,
 	iomem_read16r,
 	iomem_read32r,
@@ -253,6 +285,13 @@
 	return le16_to_cpup((u16 *)addr);
 }
 
+unsigned int ioread16be(void __iomem *addr)
+{
+	if (unlikely(INDIRECT_ADDR(addr)))
+		return iomap_ops[ADDR_TO_REGION(addr)]->read16be(addr);
+	return *((u16 *)addr);
+}
+
 unsigned int ioread32(void __iomem *addr)
 {
 	if (unlikely(INDIRECT_ADDR(addr)))
@@ -260,6 +299,13 @@
 	return le32_to_cpup((u32 *)addr);
 }
 
+unsigned int ioread32be(void __iomem *addr)
+{
+	if (unlikely(INDIRECT_ADDR(addr)))
+		return iomap_ops[ADDR_TO_REGION(addr)]->read32be(addr);
+	return *((u32 *)addr);
+}
+
 void iowrite8(u8 datum, void __iomem *addr)
 {
 	if (unlikely(INDIRECT_ADDR(addr))) {
@@ -278,12 +324,30 @@
 	}
 }
 
+void iowrite16be(u16 datum, void __iomem *addr)
+{
+	if (unlikely(INDIRECT_ADDR(addr))) {
+		iomap_ops[ADDR_TO_REGION(addr)]->write16be(datum, addr);
+	} else {
+		*((u16 *)addr) = datum;
+	}
+}
+
 void iowrite32(u32 datum, void __iomem *addr)
 {
 	if (unlikely(INDIRECT_ADDR(addr))) {
 		iomap_ops[ADDR_TO_REGION(addr)]->write32(datum, addr);
 	} else {
 		*((u32 *)addr) = cpu_to_le32(datum);
+	}
+}
+
+void iowrite32be(u32 datum, void __iomem *addr)
+{
+	if (unlikely(INDIRECT_ADDR(addr))) {
+		iomap_ops[ADDR_TO_REGION(addr)]->write32be(datum, addr);
+	} else {
+		*((u32 *)addr) = datum;
 	}
 }
 
===== include/asm-generic/iomap.h 1.3 vs edited =====
--- 1.3/include/asm-generic/iomap.h	2004-09-14 18:28:47 -05:00
+++ edited/include/asm-generic/iomap.h	2005-04-04 12:06:14 -05:00
@@ -2,6 +2,7 @@
 #define __GENERIC_IO_H
 
 #include <linux/linkage.h>
+#include <asm/byteorder.h>
 
 /*
  * These are the "generic" interfaces for doing new-style
@@ -26,11 +27,15 @@
  */
 extern unsigned int fastcall ioread8(void __iomem *);
 extern unsigned int fastcall ioread16(void __iomem *);
+extern unsigned int fastcall ioread16be(void __iomem *);
 extern unsigned int fastcall ioread32(void __iomem *);
+extern unsigned int fastcall ioread32be(void __iomem *);
 
 extern void fastcall iowrite8(u8, void __iomem *);
 extern void fastcall iowrite16(u16, void __iomem *);
+extern void fastcall iowrite16be(u16, void __iomem *);
 extern void fastcall iowrite32(u32, void __iomem *);
+extern void fastcall iowrite32be(u32, void __iomem *);
 
 /*
  * "string" versions of the above. Note that they
===== lib/iomap.c 1.6 vs edited =====
--- 1.6/lib/iomap.c	2004-10-28 02:39:50 -05:00
+++ edited/lib/iomap.c	2005-04-04 13:52:51 -05:00
@@ -58,13 +58,23 @@
 {
 	IO_COND(addr, return inw(port), return readw(addr));
 }
+unsigned int fastcall ioread16be(void __iomem *addr)
+{
+	IO_COND(addr, return inw(port), return cpu_to_be16(__raw_readw(addr)));
+}
 unsigned int fastcall ioread32(void __iomem *addr)
 {
 	IO_COND(addr, return inl(port), return readl(addr));
 }
+unsigned int fastcall ioread32be(void __iomem *addr)
+{
+	IO_COND(addr, return inl(port), return cpu_to_be32(__raw_readl(addr)));
+}
 EXPORT_SYMBOL(ioread8);
 EXPORT_SYMBOL(ioread16);
+EXPORT_SYMBOL(ioread16be);
 EXPORT_SYMBOL(ioread32);
+EXPORT_SYMBOL(ioread32be);
 
 void fastcall iowrite8(u8 val, void __iomem *addr)
 {
@@ -74,13 +84,23 @@
 {
 	IO_COND(addr, outw(val,port), writew(val, addr));
 }
+void fastcall iowrite16be(u16 val, void __iomem *addr)
+{
+	IO_COND(addr, outw(val,port), __raw_writew(be16_to_cpu(val), addr));
+}
 void fastcall iowrite32(u32 val, void __iomem *addr)
 {
 	IO_COND(addr, outl(val,port), writel(val, addr));
 }
+void fastcall iowrite32be(u32 val, void __iomem *addr)
+{
+	IO_COND(addr, outl(val,port), __raw_writel(val, addr));
+}
 EXPORT_SYMBOL(iowrite8);
 EXPORT_SYMBOL(iowrite16);
+EXPORT_SYMBOL(iowrite16be);
 EXPORT_SYMBOL(iowrite32);
+EXPORT_SYMBOL(iowrite32be);
 
 /*
  * These are the "repeat MMIO read/write" functions.

Re: iomapping a big endian area

[David S. Miller]

On Mon, 04 Apr 2005 13:57:59 -0500
James Bottomley <James.Bottomley@SteelEye.com> wrote:

> On Mon, 2005-04-04 at 16:41 +0100, David Vrabel wrote:
> > The Network Processing Engines in the Intel IXP425 are big-endian and
> > its XScale core may be run in little-endian mode. There's a bunch of
> > gotchas related to running in little-endian mode so you typically run
> > the IXP425 in big-endian mode, though.
> 
> Yes, based on feedback from Mips people and others pointing out the
> existence of the motorola rapidio bus, which is BE, I give in and agree
> that the io{read,write}{16,32}be are the way to go.
> 
> How does the attached look?

This looks fine to me.

Re: iomapping a big endian area

[Benjamin Herrenschmidt]

On Mon, 2005-04-04 at 08:59 -0500, James Bottomley wrote:
> On Mon, 2005-04-04 at 17:50 +1000, Benjamin Herrenschmidt wrote:
> > I disagree. The driver will never "know" ...
> 
> ? the driver has to know.  Look at the 53c700 to see exactly how awful
> it is.  This beast has byte and word registers.  When used BE, all the
> byte registers alter their position (to both inb and readb).

What I mean is that the driver doesn't have "know" whatever CPU/bus
combo endianness will require it to use "native" or "swapped" access.
What the driver knows is wether the device it tries to access need BE or
LE accessors.

> > I don't think it's sane. You know that your device is BE or LE and use
> > the appropriate interface. "native" doesn't make sense to me in this
> > context.
> 
> Well ... it's like this. Native means "pass through without swapping"
> and has an easy implementation on both BE and LE platforms.  Logically
> io{read,write}{16,32}be would have to do byte swaps on LE platforms.
> Being lazy, I'm opposed to doing the work if there's no actual use for
> it, so can you provide an example of a BE bus (or device) used on a LE
> platform that would actually benefit from this abstraction?

I don't think drivers will benefit from "native" vs. "swapping". Taht
means that pretty much all drivers that care will end up with #ifdef
crap to pick up the right one based on the CPU endian, and some wil get
it wrong of course. No, I _do_ thing that this should be hidden in the
implementation, and we should provide "le" and "be" accessors (le beeing
the current ones, be new ones). Which one swap or not is in the
implementation of those accessors for the architecture, but the driver
shouldn't have to care.

Ben.

Re: iomapping a big endian area

[Benjamin Herrenschmidt]

On Sat, 2005-04-02 at 21:40 -0600, James Bottomley wrote:

> Actually, ioread8be is unnecessary, but I was planning to add
> ioread16/ioread32 and iowritexx be on be variants (equivalent to
> _raw_readw et al.)
> 
> After all, the driver must know the card is BE, so the routines that
> make use of the feature are easily coded into the card, so there's no
> real need to add it to the iomem cookie.
> 
> Did anyone have a preference for the API?  I was thinking
> ioread32_native, but ioread32be is fine too.

I think we want "be" since obviously, the "ioread32" one is "le", that
makes sense to provide both and let the implementation bother with what
has to swap or not. With "native", x86 would do what ? swap or not
swap ? unclear ... with "be", at least it's clear.

The problem ? Hehe, well ... there is at least one little problem: The
way iomap "fixes" the issue of mem vs. io space in a driver could have
been used to also fix le vs. be issues. For example, an USB OHCI
controller is normally little endian. But some too-stupid-for-words HW
folks tried to be too smart on a number of embedded chips, and put a big
endian version of it. Thus the driver ends up having to support both.
Most embedded vendors just butcher the driver with #ifdef's which is
fine by me ... until you end up having _also_ a PCI bus with an
EHCI/OHCI pair on it on the same board... then you are toast.

But, I wouldn't bother too much about this case. The driver has other
issues than just IO to deal with (the DMA data structures in memory are
also endian- swapped) so I suppose the entire driver need to be somewhat
#included from a wrapper an compiled twice for different endians to get
that right...

Ben.

Re: iomapping a big endian area

[Russell King]

On Sat, Apr 02, 2005 at 09:40:39PM -0600, James Bottomley wrote:
> On Sun, 2005-04-03 at 04:10 +0100, Matthew Wilcox wrote:
> > > SPARC64 can do it in the PTEs, but we just use raw physical
> > > addresses in our I/O accessors, and in those load/store instructions
> > > we can specify the endianness.
> > 
> > Ah right.  So you'd prefer an ioread8be() interface?
> 
> Actually, ioread8be is unnecessary, but I was planning to add
> ioread16/ioread32 and iowritexx be on be variants (equivalent to
> _raw_readw et al.)

Not so.  There are two different styles of big endian.  (Lets just face
it, BE is fucked in the head anyway...)

physical bus:	31...24	23...16	15...8	7...0

BE version 1 (word invariant)
  byte access	byte 0	byte 1	byte 2	byte 3
  word access	31-24	23-16	15-8	7-0

BE version 2 (byte invariant)
  byte access	byte 3	byte 2	byte 1	byte 0
  word access	7-0	15-8	23-16	31-24

Depending on this and how your devices are wired up to such a bus, you
may need to swap bytes in a word access, or munge the byte/half word
address itself.

And guess which architecture implements *both* of these...  Grumble.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: iomapping a big endian area

[James Bottomley]

On Tue, 2005-04-05 at 08:42 +0100, Russell King wrote:
> Not so.  There are two different styles of big endian.  (Lets just face
> it, BE is fucked in the head anyway...)
> 
> physical bus:	31...24	23...16	15...8	7...0
> 
> BE version 1 (word invariant)
>   byte access	byte 0	byte 1	byte 2	byte 3
>   word access	31-24	23-16	15-8	7-0
> 
> BE version 2 (byte invariant)
>   byte access	byte 3	byte 2	byte 1	byte 0
>   word access	7-0	15-8	23-16	31-24

These are just representations of the same thing.  However, I did
deliberately elect not to try to solve this problem in the accessors.  I
know all about the register relayout, because 53c700 has to do that on
parisc.

However, I don't think it's the job of the io accessors to remap the
register locations (primarily because the remapping depends on the
documentation:  a chip that's documented as BE on BE has no remapping
required.  Hoever, the same chip on LE would need this).


> And guess which architecture implements *both* of these...  Grumble.

We have this in parisc too ...

James

Re: iomapping a big endian area

[Russell King]

On Tue, Apr 05, 2005 at 09:05:15AM -0500, James Bottomley wrote:
> On Tue, 2005-04-05 at 08:42 +0100, Russell King wrote:
> > Not so.  There are two different styles of big endian.  (Lets just face
> > it, BE is fucked in the head anyway...)
> > 
> > physical bus:	31...24	23...16	15...8	7...0
> > 
> > BE version 1 (word invariant)
> >   byte access	byte 0	byte 1	byte 2	byte 3
> >   word access	31-24	23-16	15-8	7-0
> > 
> > BE version 2 (byte invariant)
> >   byte access	byte 3	byte 2	byte 1	byte 0
> >   word access	7-0	15-8	23-16	31-24
> 
> These are just representations of the same thing.  However, I did
> deliberately elect not to try to solve this problem in the accessors.  I
> know all about the register relayout, because 53c700 has to do that on
> parisc.

They aren't.  On some of our platforms, we have to exclusive-or the address
for byte accesses with 3 to convert to the right endian-ness.

Sure, from the point of view of which byte each byte of a word represents,
it's true that they're indentical.  But as far as the hardware is concerned,
they're definitely different.

See the Intel IXP platforms for an example.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: iomapping a big endian area

[Maciej W. Rozycki]

On Tue, 5 Apr 2005, Russell King wrote:

> > > physical bus:	31...24	23...16	15...8	7...0
> > > 
> > > BE version 1 (word invariant)
> > >   byte access	byte 0	byte 1	byte 2	byte 3
> > >   word access	31-24	23-16	15-8	7-0
> > > 
> > > BE version 2 (byte invariant)
> > >   byte access	byte 3	byte 2	byte 1	byte 0
> > >   word access	7-0	15-8	23-16	31-24
> > 
> > These are just representations of the same thing.  However, I did
> > deliberately elect not to try to solve this problem in the accessors.  I
> > know all about the register relayout, because 53c700 has to do that on
> > parisc.
> 
> They aren't.  On some of our platforms, we have to exclusive-or the address
> for byte accesses with 3 to convert to the right endian-ness.

 The same with certain MIPS configurations.  And likewise you need to xor 
addresses with 2 for halfword accesses.

> Sure, from the point of view of which byte each byte of a word represents,
> it's true that they're indentical.  But as far as the hardware is concerned,
> they're definitely different.

 To clarify it a bit: both big and little endian representations are 
always the same -- it's going to a domain of the reverse endianness that 
can be done in two different ways, i.e. by preserving either bit or byte 
ordering (as described above).  Depending on the interpretation of data 
being passed you want one or the other.  That's why some systems provide 
ways of doing both kinds of accesses in hardware (e.g. the host bus to PCI 
bridge) to save CPU cycles needed for bit shuffling otherwise.

  Maciej

Re: iomapping a big endian area

[James Bottomley]

OK, I sent the patch off to Andrew.  To complete the original problem,
the attached is the patch that uses it in the parisc lasi driver
(although, actually, it sets up 53c700 to work everywhere including BE
on a LE system).

I changed some of the flags around to reflect the fact that we now have
generic BE support in the driver (rather than the more limited
force_le_on_be flag).

James

===== drivers/scsi/53c700.h 1.25 vs edited =====
--- 1.25/drivers/scsi/53c700.h	2005-04-04 09:55:44 -05:00
+++ edited/drivers/scsi/53c700.h	2005-04-04 15:39:01 -05:00
@@ -177,10 +177,10 @@
 	struct device	*dev;
 	__u32	dmode_extra;	/* adjustable bus settings */
 	__u32	differential:1;	/* if we are differential */
-#ifdef CONFIG_53C700_LE_ON_BE
+#ifdef CONFIG_53C700_BE
 	/* This option is for HP only.  Set it if your chip is wired for
 	 * little endian on this platform (which is big endian) */
-	__u32	force_le_on_be:1;
+	__u32	chip_is_be:1;
 #endif
 	__u32	chip710:1;	/* set if really a 710 not 700 */
 	__u32	burst_disable:1;	/* set to 1 to disable 710 bursting */
@@ -229,24 +229,29 @@
 /*
  *	53C700 Register Interface - the offset from the Selected base
  *	I/O address */
-#ifdef CONFIG_53C700_LE_ON_BE
-#define bE	(hostdata->force_le_on_be ? 0 : 3)
-#define	bSWAP	(hostdata->force_le_on_be)
-/* This is terrible, but there's no raw version of ioread32.  That means
- * that on a be board we swap twice (once in ioread32 and once again to 
- * get the value correct) */
-#define bS_to_io(x)	((hostdata->force_le_on_be) ? (x) : cpu_to_le32(x))
-#elif defined(__BIG_ENDIAN)
+#ifdef CONFIG_53C700_BE
+#define bE	(hostdata->chip_is_be ? 3: 0)
+#ifdef __BIG_ENDIAN
+#define	bSWAP	(!hostdata->chip_is_be)
+#else
+#define bSWAP	(hostdata->chip_is_be);
+#endif
+#define NCR_ioread32(x)	((hostdata->chip_is_be) ? ioread32be(x) : ioread32(x))
+#define NCR_iowrite32(v, x) \
+	((hostdata->chip_is_be) ? iowrite32be((v), (x)) : iowrite32((v), (x)))
+#else
+#define NCR_ioread32(x)		ioread32(x)
+#define NCR_iowrite32(v, x)	iowrite32((v), (x))
+#if defined(__BIG_ENDIAN)
 #define bE	3
 #define bSWAP	0
-#define bS_to_io(x)	(x)
 #elif defined(__LITTLE_ENDIAN)
 #define bE	0
 #define bSWAP	0
-#define bS_to_io(x)	(x)
 #else
 #error "__BIG_ENDIAN or __LITTLE_ENDIAN must be defined, did you include byteorder.h?"
 #endif
+#endif
 #define bS_to_cpu(x)	(bSWAP ? le32_to_cpu(x) : (x))
 #define bS_to_host(x)	(bSWAP ? cpu_to_le32(x) : (x))
 
@@ -460,14 +465,13 @@
 {
 	const struct NCR_700_Host_Parameters *hostdata
 		= (struct NCR_700_Host_Parameters *)host->hostdata[0];
-	__u32 value = ioread32(hostdata->base + reg);
+	__u32 value = NCR_ioread32(hostdata->base + reg);
 #if 1
 	/* sanity check the register */
-	if((reg & 0x3) != 0)
-		BUG();
+	BUG_ON((reg & 0x3) != 0);
 #endif
 
-	return bS_to_io(value);
+	return value;
 }
 
 static inline void
@@ -487,11 +491,10 @@
 
 #if 1
 	/* sanity check the register */
-	if((reg & 0x3) != 0)
-		BUG();
+	BUG_ON((reg & 0x3) != 0);
 #endif
 
-	iowrite32(bS_to_io(value), hostdata->base + reg);
+	NCR_iowrite32(value, hostdata->base + reg);
 }
 
 #endif
===== drivers/scsi/Kconfig 1.88 vs edited =====
--- 1.88/drivers/scsi/Kconfig	2005-04-04 09:55:45 -05:00
+++ edited/drivers/scsi/Kconfig	2005-04-04 15:34:40 -05:00
@@ -951,7 +951,7 @@
 	  many PA-RISC workstations & servers.  If you do not know whether you
 	  have a Lasi chip, it is safe to say "Y" here.
 
-config 53C700_LE_ON_BE
+config 53C700_BE
 	bool
 	depends on SCSI_LASI700
 	default y
===== drivers/scsi/lasi700.c 1.27 vs edited =====
--- 1.27/drivers/scsi/lasi700.c	2005-04-04 09:55:45 -05:00
+++ edited/drivers/scsi/lasi700.c	2005-04-04 15:31:19 -05:00
@@ -117,15 +117,13 @@
 
 	if (dev->id.sversion == LASI_700_SVERSION) {
 		hostdata->clock = LASI700_CLOCK;
-		hostdata->force_le_on_be = 1;
+		hostdata->chip_is_be = 0;
 	} else {
 		hostdata->clock = LASI710_CLOCK;
-		hostdata->force_le_on_be = 0;
+		hostdata->chip_is_be = 1;
 		hostdata->chip710 = 1;
 		hostdata->dmode_extra = DMODE_FC2;
 	}
-
-	NCR_700_set_mem_mapped(hostdata);
 
 	host = NCR_700_detect(&lasi700_template, hostdata, &dev->dev);
 	if (!host)

Re: iomapping a big endian area

[Christoph Hellwig]

On Mon, Apr 04, 2005 at 04:17:45PM -0500, James Bottomley wrote:
> OK, I sent the patch off to Andrew.  To complete the original problem,
> the attached is the patch that uses it in the parisc lasi driver
> (although, actually, it sets up 53c700 to work everywhere including BE
> on a LE system).
> 
> I changed some of the flags around to reflect the fact that we now have
> generic BE support in the driver (rather than the more limited
> force_le_on_be flag).

What I really don't like is that you still need ifdefs and wrappers to
support BE and LE wired chips in the same driver.  Shouldn't you set the
BE or LE flag at iomap() time and always use the same accessors?

Re: iomapping a big endian area

[James Bottomley]

On Tue, 2005-04-05 at 08:21 +0100, Christoph Hellwig wrote:
> What I really don't like is that you still need ifdefs and wrappers to
> support BE and LE wired chips in the same driver.  Shouldn't you set the
> BE or LE flag at iomap() time and always use the same accessors?

No, if you look how it works.  On parisc, it can be either BE or LE
depending on the chip wiring.  There would be a case for selecting BE or
LE by #define, but there's no case I know today where we have a driver
that would always be BE.

James