[PCI Driver]Physical address being returned as 0 in mmap

[PCI Driver]Physical address being returned as 0 in mmap

[newton mailinglist]

Hi,

I have written a PCI driver for my device. The device is an FPGA which
is configured with a design that allows it to have direct access to
memory of a host computer to which the fpga board is connected. My
device driver is responsible for translating virtual addresses to
physical addresses and sending these to the FPGA so the DMA unit in
the FPGA can directly access pages in memory using the physical
address.

I want to send about 1 MB of data to the FPGA. I currently 1st mmap
the required data file in my c program as follows :

//Open the bitstream file
	int fd = open(filename, O_RDWR);
	if (fd < 0){
		perror("molen_set : Filename Open");
		return;
	}
	
	//Get bitstream file size
	fstat(fd, &buf);
	length = buf.st_size;
	if(length == 0){
		fprintf(stderr, "\nmolen_set : Empty input file\n");
		return;
	}
	
	//Map bitstream file to memory to read it in via page faults(faster
than explicit file I/O)
	bs_map = mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, fd, 0);

This allows the file to be read in via page faults.
Then I get a suitable area from the kernel using another mmap which is
called on the PCI device :

//Get user space address of empty kernel memory of length bytes
arg = mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, htex_handle, 0);

Then I copy the data using memcpy and signal the PCI device to start
reading the data using a IOCTL

The problem I have is in the mmap implementation in the PCI driver :

static int htex_mmap(struct file * file, struct vm_area_struct * vma)
{
   vma->vm_flags |= VM_RESERVED;

   if (remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
		vma->vm_end - vma->vm_start,
		vma->vm_page_prot))
		return -EAGAIN;
	
   vma->vm_ops = &htex_vm_ops;
   htex_vma_open(vma);
	
   return 0;
}

When the driver gets the above call then I assume the kernel has
already allocated the required space and its details are in the vma
parameter. When I try to view the physical address of the virtual
address in the vma i use :

virt_to_phys(vma->vm_start)

But this gives me back the original address. A call like :
 DEBUG_MSG("htex_mmap called start:%lx, end:%lx, size:%lx, pfn:%lx, pa:%lx\n",
		vma->vm_start, vma->vm_end, vma->vm_end - vma->vm_start,
	   vma->vm_pgoff, virt_to_phys(vma->vm_start));

gives me

start:7f14b5afc000, end:7f14b5c0d000, size:111000, pfn:0, pa:f714b5afc000

I am able to get the page associated with vm_pgoff using :

struct page *pg = pfn_to_page(vma->vm_pgoff);
DEBUG_MSG("htex_mmap page: page address : %lx\n", page_address(pg));

which gives me something like :
 page address :: ffff880000000000

But this does not appear to be the correct physical address.

So how can I get the physical address of the space that the kernel
just allocated in response to the user's mmap() call on the device ?

I need the correct  physical address later to send to the FPGA as well.

Thanks
Abhi

Re: [PCI Driver]Physical address being returned as 0 in mmap

[Kyle Moffett]

On Fri, Jul 1, 2011 at 20:05, newton mailinglist
<newtonmailinglist@gmail.com> wrote:
> I have written a PCI driver for my device. The device is an FPGA which
> is configured with a design that allows it to have direct access to
> memory of a host computer to which the fpga board is connected. My
> device driver is responsible for translating virtual addresses to
> physical addresses and sending these to the FPGA so the DMA unit in
> the FPGA can directly access pages in memory using the physical
> address.
>
> [...snip...]
>
> When the driver gets the above call then I assume the kernel has
> already allocated the required space and its details are in the vma
> parameter. When I try to view the physical address of the virtual
> address in the vma i use :
>
> virt_to_phys(vma->vm_start)

You should not use virt_to_phys().  On most systems it does not work
for all memory, and on some (with an IOMMU) it would need to give a
different result depending on the device.

What you are looking for is the DMA API:

  http://lxr.linux.no/#linux/Documentation/DMA-API-HOWTO.txt
  http://lxr.linux.no/#linux/Documentation/DMA-API.txt

Specifically, you need to specify the DMA address mask of your FPGA
since not all devices can access 100% of the address space, some
are limited to 24/31/32/40-bit address spaces.  See dma_set_mask().

If you don't need fully coherent memory (as described in those docs),
then you can simply allow read() and write() on your device node and
perform DMA mapping of the userspace memory using dma_map_*().

Otherwise you will mmap() the device and use dma_alloc_coherent()
or dma_pool_*(), this is typical for things like TX queues and such.

Then make sure to insert calls to dma_sync_*() when switching
between CPU access to memory and device access to memory,
and ensure that you use appropriate IO memory barriers to order
your register reads and writes.

If you don't follow the DMA API specifications then your driver will
mysteriously fail on many platforms, even x86 if an IOMMU is present,
but particularly on processors like ARM with poor cache coherency.

Cheers,
Kyle Moffett

Re: [PCI Driver]Physical address being returned as 0 in mmap

[newton mailinglist]

ok thanks Kyle, I will try the DMA API now.

 I was wondering, when my driver gets the mmap() call then has the
 kernel already allocated memory and put its details in the vma
 parameter or do I need to allocate memory myself ?

I am already using the driver read and write functions to read/write
some other parameters to the FPGA, so I will setup the DMA in mmap()


 Cheers
 Abhi


> On Sat, Jul 2, 2011 at 3:54 AM, Kyle Moffett <kyle@moffetthome.net> wrote:
>> On Fri, Jul 1, 2011 at 20:05, newton mailinglist
>> <newtonmailinglist@gmail.com> wrote:
>>> I have written a PCI driver for my device. The device is an FPGA which
>>> is configured with a design that allows it to have direct access to
>>> memory of a host computer to which the fpga board is connected. My
>>> device driver is responsible for translating virtual addresses to
>>> physical addresses and sending these to the FPGA so the DMA unit in
>>> the FPGA can directly access pages in memory using the physical
>>> address.
>>>
>>> [...snip...]
>>>
>>> When the driver gets the above call then I assume the kernel has
>>> already allocated the required space and its details are in the vma
>>> parameter. When I try to view the physical address of the virtual
>>> address in the vma i use :
>>>
>>> virt_to_phys(vma->vm_start)
>>
>> You should not use virt_to_phys().  On most systems it does not work
>> for all memory, and on some (with an IOMMU) it would need to give a
>> different result depending on the device.
>>
>> What you are looking for is the DMA API:
>>
>>  http://lxr.linux.no/#linux/Documentation/DMA-API-HOWTO.txt
>>  http://lxr.linux.no/#linux/Documentation/DMA-API.txt
>>
>> Specifically, you need to specify the DMA address mask of your FPGA
>> since not all devices can access 100% of the address space, some
>> are limited to 24/31/32/40-bit address spaces.  See dma_set_mask().
>>
>> If you don't need fully coherent memory (as described in those docs),
>> then you can simply allow read() and write() on your device node and
>> perform DMA mapping of the userspace memory using dma_map_*().
>>
>> Otherwise you will mmap() the device and use dma_alloc_coherent()
>> or dma_pool_*(), this is typical for things like TX queues and such.
>>
>> Then make sure to insert calls to dma_sync_*() when switching
>> between CPU access to memory and device access to memory,
>> and ensure that you use appropriate IO memory barriers to order
>> your register reads and writes.
>>
>> If you don't follow the DMA API specifications then your driver will
>> mysteriously fail on many platforms, even x86 if an IOMMU is present,
>> but particularly on processors like ARM with poor cache coherency.
>>
>> Cheers,
>> Kyle Moffett
>>
>

Re: [PCI Driver]Physical address being returned as 0 in mmap

[Kyle Moffett]

Please reply inline, top-posting is unwelcome on LKML.

On Sat, Jul 2, 2011 at 11:27, newton mailinglist
<newtonmailinglist@gmail.com> wrote:
> ok thanks Kyle, I will try the DMA API now.
>
>  I was wondering, when my driver gets the mmap() call then has the
>  kernel already allocated memory and put its details in the vma
>  parameter or do I need to allocate memory myself ?

Hmm, I'm not exactly clear what you are asking here... I would need
to see your driver code to be able to give you better advice, but let
me give it a shot anyways:

The "vma" is a vm_area_struct which describes the user side of the
memory-map, and in your "mmap()" call you essentially populate that
"vma" with information about the physical CPU memory addresses.
Typically that means memory allocated with dma_alloc_coherent(),
and I think the DMA API docs have some example code for this.


> I am already using the driver read and write functions to read/write
> some other parameters to the FPGA, so I will setup the DMA in mmap()

The reason I suggest read()/write() is because you are talking about
using mmap() and then doing memcpy(), whereas if you implement
your read()/write() handlers properly then you might not need to do
a memcpy() at all.

If you're just using read()/write() for parameter data, I would switch
those to an ioctl() call and use read()/write() for your bulk data transfer.
Even if you elect to use mmap() the way you have described, you
will still need to implement some ioctl() calls to allow your userspace
program to trigger the appropriate dma_sync_*() API calls on the
memory when communicating with the hardware.

Ideally, for streaming large amounts of data out of a device, you
would call read() from the device onto a large multiple-page buffer
in your userspace program.  The userspace addresses get passed
down into your kernel driver, and you use the appropriate APIs to
get the page structs for those addresses and directly DMA-map
those pages.

That means that in the best case your device will read directly into
user memory.  If the user memory is not accessible to your device
then the kernel will use the "swiotlb" driver to bounce-buffer it for
you.

Cheers,
Kyle Moffett

Re: [PCI Driver]Physical address being returned as 0 in mmap

[newton mailinglist]

[-- Attachment #1: Type: text/plain, Size: 9818 bytes --]

ok yeah I think I am getting the hang of it now.

Well let me explain some more about what I am doing . This is going to
be a bit long as I need to explain my work a bit to explain what the
issue I am facing is.  My research involves accelerating software
functions of a c program, in hardware(thus the fpga). Our platform is
unique because it allows the fpga to have direct access to the main
memory of its host computer.

The way I accelerate a function is I make the user mark it using a
#pragma. Then using a compiler pass I remove this function's code and
replace it by code to send the function parameters to the fpga,
execute it there and read the values back.

Say we have a program like the following :
__a t t r i b u t e__  ( ( us e r ( ” r e p l a c e ” ) ) )
int foo ( int a ){ int b ;
  . . .
  . . .
  return b ;
}
int main ( void){
  return foo ( 0 ) ;
}

Then when the program is compiled the function body is replaced by the
following code :

int foo ( int a ){
   int b ;
   htex_write ( a , 0 ) ;
   htex_execute ( ) ;
   b = htex_read ( 1 ) ;
   return b ;
}

int main(void){
  htex_open();
  return foo(0);
}

The htex_write() invokes the device driver write function(you can see
molen_htex.c if you are interested, its attached). So this causes the
a parameter to be written to a memory mapped I/O region from where the
parameter goes to the fpga.  The htex_execute() causes a IOCTL call
which actually sets a control register in the FPGA to signal it to
start executing(again using memory mapped I/O). At this point the
process it put to sleep and thus the c program is stalled at the
htex_execute() till woken up.

The htex_*() functions are just wrappers for invoking device driver
operations.(the device driver file is also attached). Once done, the
FPGA interrupts the OS and the driver handles the interrupt. It wakes
up the sleeping process. Then htex_read() reads back the parameter and
puts it in the variable b. The software function can now return. The
reads and write to the fpga actually causes data to be written to and
read from BRAM inside the fpga(we call it exchange registers, but that
is just mundane hardware details)

program -> wrapper functions -> device driver -> fpga

So as you can see , I have quite a bit of working code setup to read
and write to the fpga from software(compiler passes , device driver
code etc).
Recently I have been trying to send some configuration data to the fpga.

This involves calling the htex_set() function defined in molen_htx.c
The function opens the fpga configuration file(the bitstream) and
memory maps it in the fpga. It then sends the address of the memory
mapped region to the fpga via an ioctl call(ioctl calls are handled in
htex_ioctl() in the driver).

Once the fpga receives this address it sends an interrupt to the OS to
translate the address to a physical address which the fpga can then
send to the dma unit. This virtual to physical address translation is
done in the driver. So this address translation interrupt is different
from an execution completion interrupt and there is code in place in
the interrupt handler to distinguish these 2 types of interrupts
properly.  The actual address translation happens in
translate_address() defined in the device driver file. The address
translation involves pinning the requested user page to the memory
using get_user_pages() and then getting the physical address using
pci_map_page(). This works because the calling process is asleep when
this happens.

Note that even while executing a C function which is accessing a large
amount of data from memory, the FPGA only gets the virtual address of
the array(in the exchange registers) and it requests a translation to
the physical address, which is sent to the dma unit. So I decided to
use the same process for sending the configuration data as well.

The entire process works at the moment but the only issue is the C
program is stalled when configuration data is being sent and I dont
want that.

So to improve the system what I did was not put the process to sleep
but to let it continue. Thus i want to  continue receiving address
translation interrupts from the fpga and then send physical addresses
to it while the C program runs.

But the address translation immediately failed when I tried that. I
thought it was due to trying to access the pages of a running process
and pin them using get_user_pages() as the driver fails at this exact
call.
(you can see the translate_address() function in the driver file to
see what I mean).

All I need to do is send about 1.1 mb of data to the device and I dont
really need the user process to synchronize because I want to copy all
the data to a buffer from where the device will read it. Its not
streaming data. Its a one time only send.

Ok, so to finally come to the point I decided to implement the mmap()
function call in the driver and copy the user data to kernel space.
The I would let the process run. Doing this allows me to translate
addresses which are in kernel space to physical addresses without
bothering to pin user pages to memory which wil cause problems if the
calling process is running at the time.

This is how I modified the htex_set(), to copy all the data into
kernel space(I copy all the data using memcpy(), so you can see why I
was talking about memcpy() before) :

void molen_set(const char* filename)
{
	int fd = open(filename, O_RDWR);
	int length = 0;
	char *arg, *bs_map;
	struct stat buf;

	if (fd < 0)
	{
		perror("molen_set : Filename Open");
		return;
	}
	fstat(fd, &buf);
	length = buf.st_size;
	if(length == 0)
	{
		fprintf(stderr, "\nmolen_set : Empty input file\n");
		return;
	}
	
	//Map bitstream file to memory to read it in via page faults(faster
than explicit file I/O)
	bs_map = mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, fd, 0);
	printf("molen_set : Bitstream mapped to = %p\n", bs_map);
	printf("molen_set : Bistream size = %d\n", length);
	
	//Get user space access to empty kernel memory of length bytes
	arg = mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, htex_handle, 0);
	printf("molen_set : Kernel buffer user space address = %p\n", arg);
	
	//Copy bs data to kernel space buffer : this is when the page faults
actually occur ?
	memset(arg, 0, length);
	memcpy(arg, bs_map, length);


	printf("molen_set : ioctl called\n");
	ioctl(htex_handle, HTEX_IOSET, arg);
	printf("molen_set : ioctl returned\n");

	munmap(arg, length);
	munmap(bs_map, length);
}

The mmap() call to the device driver lands in the function htex_mmap()
defined in htex_driver.c which is where I want to do the dma setup as
I am already using htex_read() and htex_write() for function
parameters are explained before.

So from our dicussion so far I think what might work is if I read in
the user file and copy it to a DMA buffer and then send the address of
this buffer to the fpga. To copy the data I was using memcpy() inside
the modified htex_set()...see above. I can perhaps continue to copy
the user data to the dma buffer using memcpy().

Hopefully once copied, I can let the C program proceed, and the DMA
api will give me a physical address which I can send to the fpga to
read the buffer from memory.

Thanks,
Abhi


On Sat, Jul 2, 2011 at 6:24 PM, Kyle Moffett <kyle@moffetthome.net> wrote:
> Please reply inline, top-posting is unwelcome on LKML.
>
> On Sat, Jul 2, 2011 at 11:27, newton mailinglist
> <newtonmailinglist@gmail.com> wrote:
>> ok thanks Kyle, I will try the DMA API now.
>>
>>  I was wondering, when my driver gets the mmap() call then has the
>>  kernel already allocated memory and put its details in the vma
>>  parameter or do I need to allocate memory myself ?
>
> Hmm, I'm not exactly clear what you are asking here... I would need
> to see your driver code to be able to give you better advice, but let
> me give it a shot anyways:
>
> The "vma" is a vm_area_struct which describes the user side of the
> memory-map, and in your "mmap()" call you essentially populate that
> "vma" with information about the physical CPU memory addresses.
> Typically that means memory allocated with dma_alloc_coherent(),
> and I think the DMA API docs have some example code for this.
>
>
>> I am already using the driver read and write functions to read/write
>> some other parameters to the FPGA, so I will setup the DMA in mmap()
>
> The reason I suggest read()/write() is because you are talking about
> using mmap() and then doing memcpy(), whereas if you implement
> your read()/write() handlers properly then you might not need to do
> a memcpy() at all.
>
> If you're just using read()/write() for parameter data, I would switch
> those to an ioctl() call and use read()/write() for your bulk data transfer.
> Even if you elect to use mmap() the way you have described, you
> will still need to implement some ioctl() calls to allow your userspace
> program to trigger the appropriate dma_sync_*() API calls on the
> memory when communicating with the hardware.
>
> Ideally, for streaming large amounts of data out of a device, you
> would call read() from the device onto a large multiple-page buffer
> in your userspace program.  The userspace addresses get passed
> down into your kernel driver, and you use the appropriate APIs to
> get the page structs for those addresses and directly DMA-map
> those pages.
>
> That means that in the best case your device will read directly into
> user memory.  If the user memory is not accessible to your device
> then the kernel will use the "swiotlb" driver to bounce-buffer it for
> you.
>
> Cheers,
> Kyle Moffett
>

[-- Attachment #2: molen_htx.c --]
[-- Type: text/x-csrc, Size: 2598 bytes --]

#include <stdint.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include "htex_fcntl.h"
#include <string.h>

int htex_handle = 0;

int molen_htx_init(void)
{
	int handle = 0;
	if(htex_handle)
		return 0;
	handle = open("/dev/htex",O_RDWR);
	if(handle < 0)
	{
		perror("molen_htx_init : open");
		return 1;
	}
	htex_handle = handle;
	return 0;
}

void molen_write(unsigned long arg, unsigned long index)
{
	lseek(htex_handle, index*sizeof(unsigned long), SEEK_SET);
	write(htex_handle, (void *)&arg, sizeof(unsigned long));
}

unsigned long molen_read(unsigned long index)
{
	unsigned long arg;
	lseek(htex_handle, index*sizeof(unsigned long), SEEK_SET);
	read(htex_handle, (void *)&arg, sizeof(unsigned long));
	return arg;
}

void molen_execute(void)
{
	ioctl(htex_handle, HTEX_IOEXECUTE);	
}

void molen_set(const char* filename)
{
	int fd = open(filename, O_RDWR);
	int length = 0;
	char *arg, *bs_map;
	struct stat buf;

	if (fd < 0)
	{
		perror("molen_set : Filename Open");
		return;
	}
	fstat(fd, &buf);
	length = buf.st_size;
	if(length == 0)
	{
		fprintf(stderr, "\nmolen_set : Empty input file\n");
		return;
	}
	
	//Map bitstream file to memory to read it in via page faults(faster than explicit file I/O)
	bs_map = mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, fd, 0);
	printf("molen_set : Bitstream mapped to = %p\n", bs_map);
	printf("molen_set : Bistream size = %d\n", length);
	
	//Get user space access to empty kernel memory of length bytes
	arg = mmap(NULL, length, PROT_WRITE | PROT_READ, MAP_PRIVATE, htex_handle, 0);
	printf("molen_set : Kernel buffer user space address = %p\n", arg);
	
	//Copy bs data to kernel space buffer : this is when the page faults actually occur ?
	memset(arg, 0, length);
	memcpy(arg, bs_map, length);


	printf("molen_set : ioctl called\n");
	ioctl(htex_handle, HTEX_IOSET, arg);
	printf("molen_set : ioctl returned\n");

	munmap(arg, length);
	munmap(bs_map, length);
}

char* molen_elfset(char* start)
{
	printf("molen_elfset : Bitstream = %p\n", start);
	

	printf("molen_elfset : ioctl called\n");
	ioctl(htex_handle, HTEX_IOSET, start);
	printf("molen_elfset : ioctl returned\n");
	//ioctl(htex_handle, HTEX_IORESET);
	//printf("molen_elfset : ioctl reset issued\n");	
	start = NULL;
	
	return start;
	
}

void molen_reset(void)
{
	ioctl(htex_handle, HTEX_IORESET);
}

void molen_close(void)
{
	close(htex_handle);
	htex_handle = 0;
}


[-- Attachment #3: htex_driver.c --]
[-- Type: text/x-csrc, Size: 25422 bytes --]


#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
//#include <linux/ioport.h>
//#include <asm/io.h>
#include <linux/interrupt.h>
//#include <linux/cdev.h>
//#include <linux/types.h>
//#include <linux/fs.h>
//#include <asm/uaccess.h>
//#include <linux/slab.h>
//#include <linux/ioctl.h>
//#include <linux/wait.h>
//#include <linux/mm.h>
#include <linux/pagemap.h>
//#include <asm/atomic.h>
#include <linux/rwsem.h>
#include "htex_fcntl.h"
#include "htex_driver.h"
 

//MODULE_LICENSE("GPLv2");

#define DEBUG

#ifdef DEBUG
#define DEBUG_MSG(...) printk( KERN_INFO "htex: " __VA_ARGS__)
#else
#define DEBUG_MSG(a,...)
#endif

#define ERROR_MSG(...) printk( KERN_WARNING "htex: error, " __VA_ARGS__)

static wait_queue_head_t wait_queue;


static atomic_t available = ATOMIC_INIT(1);

static struct pci_device_id ids[] ={
	{ PCI_DEVICE(0x0007,0x0009)},
	{0,},
};

static struct pci_driver pci_driver = {
	.name = "htex_driver",
	.id_table = ids,
	.probe = probe,
	.remove = remove,
};

static struct file_operations htex_fops = {
	.owner   = THIS_MODULE,
	.llseek  = htex_llseek,
	.read    = htex_read,
	.write   = htex_write,
	.open    = htex_open,
	.release = htex_release,
	.ioctl   = htex_ioctl,
	.mmap    = htex_mmap
};

unsigned long j4,j2, j3;

MODULE_DEVICE_TABLE(pci, ids);

/**
register htex_driver with kernel
*/
static int __init test_init(void)
{
	DEBUG_MSG("init\n");
	//register pci driver
	return pci_register_driver(&pci_driver);
}

/**
 * unregister pci driver from kernel
 */
static void __exit test_exit(void)
{
	DEBUG_MSG("exit\n");
	//unregister driver
	pci_unregister_driver(&pci_driver);
}

/**
 * handle interupt from device
 */
irqreturn_t interrupt_handler( int irq, void *dev_id)
{
	struct htex_dev_t *htex_dev = (struct htex_dev_t *) dev_id;
	//long arg = ((long *)htex_dev->bar2->bar)[1];
	int status = htex_dev->bar2->bar[0];
	//DEBUG_MSG("addr = %lx\n", arg);
	
	htex_dev->irq_count++;
	rdtscll(j4);
	//htex_dev->wq_count++;
	//in the future maybe this will be needed.
	DEBUG_MSG("Interrupt received count = %d.\n", htex_dev->irq_count);
	//	wake_up_interruptible(&wait_queue);

	if(status == 1)
	{
		DEBUG_MSG("CCU execution done\n");
	}
	else if (status == 2)
		DEBUG_MSG("Address interrupt\n");
	else if(status == 4)
	  	DEBUG_MSG("SET interrupt\n");
	else
		DEBUG_MSG("unknonw interrupt %d\n", status);
	schedule_work(&htex_dev->ws);
	
	return IRQ_HANDLED;
}

void print_tlb_entry(int index, struct htex_dev_t *htex_dev)
{
	unsigned int *addr = (unsigned int *)htex_dev->bar4->bar;
	DEBUG_MSG("TLB Entry = %8x%8x%8x\n", ioread32(addr + index*4), ioread32(addr + index*4+1), ioread32(addr + index*4+2));
}

void htex_interrupt_tasklet(struct work_struct *work)
{
	struct htex_dev_t *htex_dev = container_of(work, struct htex_dev_t, ws);
	int status = 0;
	status = htex_dev->bar2->bar[0];
	//signal that interrupt is handled
	rdtscll(j3);
	iowrite32(HTEX_IRQ_HANDLED_C, htex_dev->bar2->bar);
	if(status == 1)
	{
		DEBUG_MSG("Execution done\n");
		htex_dev->done = 1;
		wake_up_interruptible(&wait_queue);
	}
	else if(status == 2)
	{
		long arg = ((long *)htex_dev->bar2->bar)[1];
		DEBUG_MSG("Address translation request\n");
		DEBUG_MSG("Address = %lx\n", arg);
		if (!translate_address(arg, htex_dev))
		{ 
		  DEBUG_MSG("Address Translation of the address %lx has failed !!\n", arg);
		  return;
		}
		else
		{
			//do this for now until the tlb is fixed
			//iowrite32((unsigned int)htex_dev->entries[virt_to_index(arg)].hw_addr, ((long *)htex_dev->bar2->bar) + 1);
			//DEBUG_MSG("xxxx hw addr = %lx\n", (unsigned long)htex_dev->entries[virt_to_index(arg)].hw_addr); 
		}
	}
	else if(status == 4)
	{
	    DEBUG_MSG("Configuration Complete\n");	   
	    htex_dev->set_done = 1;
	    //wake_up_interruptible(&wait_queue);
	}
	else
	{
		DEBUG_MSG("unknown interrupt: %d\n", status);
	}
	rdtscll(j2);
	DEBUG_MSG("%ld cycles %ld cycles\n", j3-j4, j2-j3);
}

/**
 * request memory region bar and remaps the region.
 * returns 0 on error or the address a struct containing information on the remapped region.
 */
bar_t *get_bar(struct pci_dev *dev, int bar)
{
	unsigned long int begin = (unsigned long int)pci_resource_start(dev, bar);
	unsigned long int size = (unsigned long int)pci_resource_len(dev, bar);
	char *bar_map = 0;
	struct resource *io_mem;
	bar_t *bar_st = kzalloc(sizeof(bar_t), GFP_KERNEL);
	if(bar_st == 0)
	{
		ERROR_MSG("could not allocate bar struct\n");
		goto err_alloc;
	}
	io_mem = request_mem_region(begin, size, "htex");
	if (io_mem == 0)
	{
		ERROR_MSG("bar %d unavailable\n", bar);
		goto err_req;
	}
	//remap the bar address space so it can be accessed
	bar_map = (char *)ioremap_nocache(begin, size);
	if(bar_map == 0)
	{
		ERROR_MSG("could not map bar %d\n", bar);
		goto err_map;
	}
	bar_st->begin = begin;
	bar_st->size = size;
	bar_st->bar = bar_map;
	return bar_st;

	err_map:
	release_mem_region(begin, size);
	err_req:
	kfree(bar_st);
	err_alloc:
	return 0;
	
}

/**
 *unmaps the memory region mapped in bar and frees the allocated memory of the bar struct.
 */
void unget_bar(bar_t *bar)
{
	iounmap(bar->bar);
	release_mem_region(bar->begin, bar->size);
	kfree(bar);
}

/**
 * this function initializes the pci device and makes it ready to be used.
 * it maps the bar address space and allocates a dma buffer that the device can access.
 */
int __devinit probe(struct pci_dev *dev, const struct pci_device_id *id)
{
	int result;
	struct htex_dev_t *htex_dev;


	DEBUG_MSG("probe\n");
	
	init_waitqueue_head(&wait_queue);
	
	//allocate the structure to store driver data
	htex_dev = kzalloc(sizeof (struct htex_dev_t), GFP_KERNEL);
	if(!htex_dev)
		goto err_out;
	
	//since kzalloc is used everything is already set to 0
	htex_dev->irq_count = 0;
	htex_dev->dev = dev;
	htex_dev->entries = 0;
	htex_dev->tlb_size = 0;
	//htex_dev->wq_count = 0;

	result =  pci_enable_device(dev);
	if(result != 0)
		goto err_enable_device;
	
	htex_dev->bar0 = get_bar(dev, 0);
	if(htex_dev->bar0 == 0)
	{
		goto err_get_bar0;
	}
	
	htex_dev->bar2 = get_bar(dev, 2);
	if(htex_dev->bar2 == 0)
	{
		goto err_get_bar2;
	}
	
	htex_dev->bar4 = get_bar(dev, 4);
	if(htex_dev->bar4 == 0)
	{
		goto err_get_bar4;
	}
	htex_dev->tlb_size = htex_dev->bar4->size/16;
	
	//create irq number for htx device
	htex_dev->irq = ht_create_irq(dev, 0);
	if (htex_dev->irq < 0)
	{
		ERROR_MSG("create irq\n");
		goto err_create_irq;
	}
	//register isr for irq number
	result = request_irq(htex_dev->irq, interrupt_handler, 0, "htex", htex_dev);
	if (result)
	{
		ERROR_MSG("request irq\n");
		goto err_request_irq;
	}
	
	//checks and sets device addressable map	
	if (!pci_set_consistent_dma_mask(dev, DMA_64BIT_MASK))
	{
		DEBUG_MSG("using 64bit DMA addressing\n");
	}
	else if (!pci_set_consistent_dma_mask(dev, DMA_32BIT_MASK))
	{
		DEBUG_MSG("using 32bit DMA addressing\n");
	}
	else 
	{
		ERROR_MSG("no suitable DMA available\n");
		goto err_set_consistent;
	}
	

	result = alloc_chrdev_region(&htex_dev->devno, 0, 1, "htex");
	if(result < 0)
	{
		ERROR_MSG("cannot get major nr\n");
		goto err_get_major;
	}
	//initialize fields in structure
	cdev_init(&htex_dev->cdev, &htex_fops);
	htex_dev->cdev.owner = THIS_MODULE;
	
	//store the structure so it can be accessed later
	pci_set_drvdata(dev, htex_dev);
	
	INIT_WORK(&htex_dev->ws, htex_interrupt_tasklet);
	clear_tlb(htex_dev);
	//finally add char device so it can be used
	result = cdev_add(&htex_dev->cdev, htex_dev->devno, 1);
	if (result)
	{
		ERROR_MSG("adding char device\n");
		goto err_cdev_add;
	}
	return 0;
	//if there was an error undo all the steps that had already completed before returning
	err_cdev_add:
	unregister_chrdev_region(htex_dev->devno, 1);
	
	err_get_major:
	
	err_set_consistent:
	free_irq(htex_dev->irq, dev);
	
	err_request_irq:
	ht_destroy_irq(htex_dev->irq);

	err_create_irq:
	unget_bar(htex_dev->bar4);
	
	err_get_bar4:
	unget_bar(htex_dev->bar2);
	err_get_bar2:
	unget_bar(htex_dev->bar0);
	err_get_bar0:
	pci_disable_device(dev);

	err_enable_device:
	kfree(htex_dev);

	err_out:
	return -ENODEV;
}

/**
 * undo everything that was done in htex_probe
 */
void __devexit remove(struct pci_dev *dev)
{
	struct htex_dev_t *htex_dev = pci_get_drvdata(dev);
	DEBUG_MSG("remove\n");
	cdev_del(&htex_dev->cdev);
	unregister_chrdev_region(htex_dev->devno, 1);
	unget_bar(htex_dev->bar4);
	unget_bar(htex_dev->bar2);
	unget_bar(htex_dev->bar0);
	free_irq(htex_dev->irq, htex_dev);
	ht_destroy_irq(htex_dev->irq);
	pci_disable_device(dev);
	kfree(htex_dev);
}

/** 
 * Handle mmap of the htex example device : added by abhi
 */
void htex_vma_open(struct vm_area_struct *vma)
{
	printk(KERN_NOTICE "htex VMA open, virt %lx, phys %lx\n",
	vma->vm_start, vma->vm_pgoff << PAGE_SHIFT);
}

void htex_vma_close(struct vm_area_struct *vma)
{
	printk(KERN_NOTICE "htex VMA close.\n");
}

static struct vm_operations_struct htex_vm_ops = {
	.open  = htex_vma_open,
	.close = htex_vma_close,
};

static int htex_mmap(struct file * file, struct vm_area_struct * vma)
{
   DEBUG_MSG("htex_mmap called start:%lx, end:%lx, size:%lx, pfn:%lx, pa:%lx\n",
		vma->vm_start, vma->vm_end, vma->vm_end - vma->vm_start,
	   vma->vm_pgoff, virt_to_phys(vma->vm_start));
	
   struct page *pg = pfn_to_page(vma->vm_pgoff);

   DEBUG_MSG("htex_mmap page: pg : %lx\n", page_address(pg));
   vma->vm_flags |= VM_RESERVED;
  
   if (remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
		vma->vm_end - vma->vm_start,
		vma->vm_page_prot))
		return -EAGAIN;
	
   vma->vm_ops = &htex_vm_ops;
   htex_vma_open(vma);
	
   return 0;
}


/** 
 * Handle open of the htex example device.
 */
static int htex_open(struct inode *inode, struct file *filp)
{
	int minor_nr;
	//get struct with driver info
	struct htex_dev_t *htex_dev = container_of(inode->i_cdev, struct htex_dev_t, cdev);
	minor_nr = iminor(inode);

	DEBUG_MSG("open\n");
	if (minor_nr != 0)
	{
		ERROR_MSG("Note that only minor number 0 is valid!\n");
		return -ENODEV;
	}
	//check if device is already open
	if (! atomic_dec_and_test(&available))
	{
		atomic_inc(&available);
		ERROR_MSG("device already opened\n");
		return -EBUSY;
	}
	//store struct for open and close
	filp->private_data = htex_dev;
	htex_dev->tsk = current;
	htex_dev->entries = kzalloc(htex_dev->tlb_size * sizeof(struct tlb_entry), GFP_KERNEL);
	if(!htex_dev->entries)
	{
		ERROR_MSG("could not allocate pages array\n");
		return -ENODEV;
	}
	return 0;
}

/** 
 * Handle close of the htex example device.
 */
static int htex_release(struct inode *inode,struct file *filp)
{
	int minor_nr = iminor(inode);
	struct htex_dev_t *htex_dev = container_of(inode->i_cdev, struct htex_dev_t, cdev);
	if (minor_nr!=0)     
	{
		ERROR_MSG("only minor 0 is valid\n");
		return -ENODEV;
	}
	kfree(htex_dev->entries);
	DEBUG_MSG("close\n");
	atomic_inc(&available);
	return 0;
}

static loff_t htex_llseek(struct file *filp, loff_t off, int whence)
{
	struct htex_dev_t *htex_dev = filp->private_data;
	loff_t newpos;

	switch(whence)
	{
		case 0:
		newpos = off;
		break;

		case 1:
		newpos = filp->f_pos + off;
		break;

		case 2:
		newpos = htex_dev->bar0->size + off;
		break;

		default:
		return -EINVAL;
	}
	if (newpos < 0) return -EINVAL;
	if (newpos >= htex_dev->bar0->size) return -EINVAL;
	filp->f_pos = newpos;
	return newpos;
}

/**
 * Read reads data from the bar0 address space of the htex device
 */
static ssize_t htex_read(struct file *filp, char *buf, size_t count, loff_t *f_pos)
{
	struct htex_dev_t *htex_dev = filp->private_data;
	ssize_t retval = 0;
	DEBUG_MSG("read\n");
	//if f_pos is grater than the size of buffer we cannot read from it
	if (*f_pos >= htex_dev->bar0->size)
		goto out;
	//make sure that if we read count bytes we dont go over the size of the buffer
	if(*f_pos + count > htex_dev->bar0->size)
		count = htex_dev->bar0->size - *f_pos;
	//copy data over to user and check for error
	if(copy_to_user(buf, htex_dev->bar0->bar + (long)*f_pos, count))
	{
		retval = -EFAULT;
		ERROR_MSG("read\n");
		goto out;
	}
	//update f_pos and return number of bytes read
	*f_pos += count;
	retval = count;
	out:
	return retval;
}

/**
 * Write writes to the bar0 address space of the htex device ignoring f_pos
 */
static ssize_t htex_write(struct file *filp, const char *buf, size_t count, loff_t *f_pos)
{
	struct htex_dev_t *htex_dev = filp->private_data;
	ssize_t retval = 0;
	DEBUG_MSG("write\n");
	//if f_pos is grater than the size of buffer we cannot write to it
	if (*f_pos >= htex_dev->bar0->size)
		goto out;
	//make sure that if we write count bytes we dont go over the size of the buffer
	if(*f_pos + count > htex_dev->bar0->size)
		count = htex_dev->bar0->size - *f_pos;
	//copy data over from user and check for error
	if(copy_from_user(htex_dev->bar0->bar+(long)*f_pos, buf, count))
	{
		retval = -EFAULT;
		ERROR_MSG("write\n");
		goto out;
	}
	//update f_pos and return number of bytes written
	*f_pos += count;
	retval = count;
	out:
	return retval;
}

/**
 * unmap and release the page stored at index in the array of tlb entries
 */
void release_page(struct htex_dev_t *htex_dev, unsigned int index)
{
	pci_unmap_page(htex_dev->dev, htex_dev->entries[index].hw_addr, PAGE_SIZE, DMA_BIDIRECTIONAL);
	//SetPageDirty(htex_dev->entries[index].page);
	page_cache_release(htex_dev->entries[index].page);
	htex_dev->entries[index].page = 0;
	//set valid flag to 0 on device
	iowrite32(0, ((unsigned int *)htex_dev->bar4->bar)+index*4 + 2);
}

void release_all_pages(struct htex_dev_t *htex_dev)
{
	int i = 0;
	DEBUG_MSG("Released all pages\n");
	for(i = 0; i < htex_dev->tlb_size; i++)
	{
		if(htex_dev->entries[i].page)
		{
			release_page(htex_dev, i);
		}
	}
}

/**
 * constructs the tlb entry (tag hwaddress and flags) and writes it to the device
 */
void update_htx_tlb(unsigned long int index, unsigned long int tag, unsigned long int hw_addr, struct htex_dev_t *htex_dev)
{
	unsigned long int pfn;
	unsigned int entry0 = 0, entry1 = 0, entry2 = 0;
	//write tlb entry to htex_device
	pfn = (hw_addr >> PAGE_SHIFT) & 0xFFFFFFFF;
	
	entry0 = (unsigned int)(pfn & 0x00000000ffffffff);
	entry1 = (unsigned int)((pfn | (tag << 40)) >> 32);
	entry2 = (unsigned int)(tag >> 24) | ((long)1 << 31);
	
	iowrite32(entry0, ((unsigned int *)htex_dev->bar4->bar)+index*4);
	iowrite32(entry1, ((unsigned int *)htex_dev->bar4->bar)+index*4 + 1);
	iowrite32(entry2, ((unsigned int *)htex_dev->bar4->bar)+index*4 + 2);
	return;
}

/**
 * write 0 to every tlb entry
 */
void clear_tlb(struct htex_dev_t *htex_dev)
{
	int i = 0;
	for(i = 0; i < htex_dev->tlb_size; i++)
	{
		iowrite32(0, ((unsigned int *)htex_dev->bar4->bar)+i*4);
		iowrite32(0, ((unsigned int *)htex_dev->bar4->bar)+i*4 + 1);
		iowrite32(0, ((unsigned int *)htex_dev->bar4->bar)+i*4 + 2);
	}
}
void htx_dump_tlb(struct htex_dev_t *htex_dev)
{
	int i;
	for (i = 0; i < htex_dev->bar4->size/16; i++)
	{
		print_tlb_entry(i, htex_dev);
	}
}

/**
 *get the page belonging to address, lock it in memory and get the bus address of the page
 *return 0 on failure and 1 on success
 */
int translate_address(unsigned long int virt_addr, struct htex_dev_t *htex_dev)
{
	int result;
	unsigned long int translated;
	unsigned long int index = 0;
	unsigned long int tag = 0;
	struct page *page;	
	
	index = virt_to_index(virt_addr);
	tag = virt_to_tag(virt_addr);

	DEBUG_MSG("translate_address: virt_addr = %lx, index=%lx, tag=%lx\n", virt_addr, index, tag);
	
	//check if index already has a valid entry and if so
	//release this entry before replacing it
	if(htex_dev->entries[index].page)
		release_page(htex_dev, index);
	
	
	//init_rwsem(&sem);
	if(down_read_trylock(&htex_dev->tsk->mm->mmap_sem))
		DEBUG_MSG("translate_address: lock was granted!\n");
	else{
		DEBUG_MSG("translate_address: A Lock was not granted, skipping address translation\n");
		goto cleanup;
	}
	
	//get page
/*	result = get_user_pages(htex_dev->tsk, htex_dev->tsk->mm, virt_addr, 1, 0, 0, &page, NULL);
	DEBUG_MSG("translate_address: result = %d\n", result);
	if (result <= 0)
	{
		ERROR_MSG("translate_address: Unable to get page\n");
		//htx_dump_tlb(htex_dev);
		return 0;
	}*/

	//translated = virt_to_phys((volatile void *)virt_addr);
	page = vmalloc_to_page((void*)virt_addr);
	//get bus address of page
	translated = pci_map_page(htex_dev->dev, page, 0, PAGE_SIZE, DMA_BIDIRECTIONAL);

	DEBUG_MSG("translate_address: Translated Address: %lx\n", translated);
	
	update_htx_tlb(index, tag, translated, htex_dev);
	//update entry in array
	htex_dev->entries[index].page = page;
	htex_dev->entries[index].hw_addr = translated;
	
	cleanup:
	up_read(&htex_dev->tsk->mm->mmap_sem);
	return 1;
}

/**
 *Translates the addresses for count pages starting at start and writes them to the device
 *returns the number of pages mapped and translated
 */
int translate_range(unsigned long int start, unsigned int count, struct htex_dev_t *htex_dev)
{
	int i, map_count = 0;
	unsigned long int virt_addr = start;
	
	for(i=0; i < count; i++)
	{
		if(!translate_address(virt_addr, htex_dev))
		{
			//try to translate next address in range
			virt_addr += PAGE_SIZE;
			continue;
		}
		map_count++;
		virt_addr += PAGE_SIZE;
	}
	return map_count;
}

void prefill_tlb(struct htex_dev_t *htex_dev)
{
	int heap_count, stack_count;
	unsigned long start_heap, start_stack, end_heap;
	//struct vm_area_struct *vma_stack,*vma_heap;
	struct mm_struct *mm = htex_dev->tsk->mm; 
	DEBUG_MSG("Prefilling tlb with stack and heap addresses\n");
	/*vma_stack = find_vma(mm,mm->start_stack);
	vma_heap = find_vma(mm,mm->start_brk);*/
	start_heap = mm->start_brk;
	end_heap = mm->brk;
	start_stack = mm->start_stack;
	stack_count = mm->stack_vm;
	
	//prefill stack
	start_stack = start_stack & PAGE_MASK;
	start_stack = start_stack - ((stack_count-1)*PAGE_SIZE);
	translate_range(start_stack, stack_count, htex_dev);
	
	//prefill heap
	heap_count = (end_heap - start_heap) >> PAGE_SHIFT;
	translate_range(start_heap, heap_count, htex_dev);
}


/**
 * compare the tlb before and after execution
 */
void compare_tlb(struct htex_dev_t *htex_dev)
{
	int i = 0;
	struct tlb_entry *entries = htex_dev->entries;
	htex_dev->entries = kzalloc(htex_dev->tlb_size * sizeof(struct tlb_entry), GFP_KERNEL);
	prefill_tlb(htex_dev);
	for(i = 0; i < htex_dev->tlb_size; i++)
	{
		if(entries[i].page && (entries[i].page != htex_dev->entries[i].page || entries[i].hw_addr != htex_dev->entries[i].hw_addr))
		{
			ERROR_MSG("tlb not the same\t");
			ERROR_MSG("index = %d\n", i);
			ERROR_MSG("hw addr1 = %lx\n", (unsigned long)entries[i].hw_addr);
			ERROR_MSG("hw addr2 = %lx\n", (unsigned long)htex_dev->entries[i].hw_addr);
			release_page(htex_dev, i);
		}
	}
	kfree(htex_dev->entries);
	htex_dev->entries = entries;
}

/**
 * this ioctl is used to issue the execute, set and optionally reset instrcutions
 */
static int htex_ioctl(struct inode *inode, struct file *filp, unsigned int cmd, unsigned long arg)
{	long status;
	struct htex_dev_t *htex_dev = filp->private_data;
	int result;	
	u32 temp;
	
	DEBUG_MSG("HTEX_IORESET : %d\n",HTEX_IORESET);
	DEBUG_MSG("HTEX_IOEXECUTE : %d\n",HTEX_IOEXECUTE);
	DEBUG_MSG("HTEX_IOSET : %ld\n",HTEX_IOSET);
	DEBUG_MSG("IOCTL issued : %d\n",cmd);
	if((_IOC_TYPE(cmd) != HTEX_IOC_MAGIC) || (_IOC_NR(cmd) > HTEX_IOC_MAXNR))
  
		return -ENOTTY;

	switch (cmd)
	{
		case HTEX_IORESET:
			iowrite32(HTEX_RESET_C, htex_dev->bar2->bar);
		//	iowrite32(HTEX_IRQ_HANDLED_C, htex_dev->bar2->bar);
			break;
		case HTEX_IOEXECUTE:
			//reset the tlb and ccu
			iowrite32(HTEX_RESET_C, htex_dev->bar2->bar);
			//prefill_tlb(htex_dev);
			
			DEBUG_MSG("Waiting for reconfiguration to complete.\n");
			result = wait_event_interruptible(wait_queue, htex_dev->set_done != 0);	//wait till prefetching over
			//release_all_pages(htex_dev);
			
			DEBUG_MSG("Starting CCU EXECUTION\n");
			iowrite32(HTEX_EXECUTE_C, htex_dev->bar2->bar);			
			htex_dev->done = 0;
			result = wait_event_interruptible(wait_queue, htex_dev->done != 0);
			DEBUG_MSG("TLB state %d\n", ioread32(htex_dev->bar2->bar+16));
			
			status = ioread32(htex_dev->bar2->bar+0);
			DEBUG_MSG("IOEXECUTE DD 0 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+4);
			DEBUG_MSG("IOEXECUTE DD 1 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+8);
			DEBUG_MSG("IOEXECUTE DD 2 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+12);
			DEBUG_MSG("IOEXECUTE DD 3 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+16);
			DEBUG_MSG("IOEXECUTE DD 4 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+20);
			DEBUG_MSG("IOEXECUTE DD 5 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+24);
			DEBUG_MSG("IOEXECUTE DD 6 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+28);
			DEBUG_MSG("IOEXECUTE DD 7 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+32);
			DEBUG_MSG("IOEXECUTE DD 8 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+36);
			DEBUG_MSG("IOEXECUTE DD 9 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+40);
			DEBUG_MSG("IOEXECUTE DD 10 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+44);
			DEBUG_MSG("IOEXECUTE DD 11 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+48);
			DEBUG_MSG("IOEXECUTE DD 12 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+52);
			DEBUG_MSG("IOEXECUTE DD 13 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+56);
			DEBUG_MSG("IOEXECUTE DD 14 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+60);
			DEBUG_MSG("IOEXECUTE DD 15 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+64);					
			
			//htx_dump_tlb(htex_dev);
			release_all_pages(htex_dev);
			htex_dev->done = 0;
			if (result != 0)
				return -ERESTARTSYS;
			break;
		case HTEX_IOSET:			//Modified for prefetch
			//reset the tlb and ccu
			iowrite32(HTEX_RESET_C, htex_dev->bar2->bar);
			//iowrite32(arg, htex_dev->bar2->bar+8);
			*(((u64 *)htex_dev->bar2->bar) + 1) = arg;
			//prefill_tlb(htex_dev);
			temp = ioread32(htex_dev->bar2->bar+16);
			
			//Verify bs data
			DEBUG_MSG("B.S. data = %lx\n",*((unsigned long *)arg));	
			DEBUG_MSG("B.S. data = %lx\n",*((unsigned long *)arg+8));
			
			DEBUG_MSG("icap before state = %d, busy = %d\n",(temp >> 3) & 0x7, (temp >> 6)&0x1);
			DEBUG_MSG("icap before data = %x\n",ioread32(htex_dev->bar2->bar+24));
			
			DEBUG_MSG("IN SET\n");
			iowrite32(HTEX_SET_C, htex_dev->bar2->bar);			
			
			//result = wait_event_interruptible(wait_queue, htex_dev->done != 0);	//removed to allow c program to continue
			temp = ioread32(htex_dev->bar2->bar+16);
			
			DEBUG_MSG("icap after state = %d, busy = %d\n",(temp >> 3) & 0x7, (temp >> 6)&0x1);
			DEBUG_MSG("icap after data = %x\n",ioread32(htex_dev->bar2->bar+24));			
			
			status = ioread32(htex_dev->bar2->bar+0);
			DEBUG_MSG("IOSET DD 0 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+4);
			DEBUG_MSG("IOSET DD 1 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+8);
			DEBUG_MSG("IOSET DD 2 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+12);
			DEBUG_MSG("IOSET DD 3 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+16);
			DEBUG_MSG("IOSET DD 4 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+20);
			DEBUG_MSG("IOSET DD 5 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+24);
			DEBUG_MSG("IOSET DD 6 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+28);
			DEBUG_MSG("IOSET DD 7 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+32);
			DEBUG_MSG("IOSET DD 8 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+36);
			DEBUG_MSG("IOSET DD 9 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+40);
			DEBUG_MSG("IOSET DD 10 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+44);
			DEBUG_MSG("IOSET DD 11 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+48);
			DEBUG_MSG("IOSET DD 12 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+52);
			DEBUG_MSG("IOSET DD 13 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+56);
			DEBUG_MSG("IOSET DD 14 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+60);
			DEBUG_MSG("IOSET DD 15 = %.8lx\n", status);
			status = ioread32(htex_dev->bar2->bar+64);			
			
			//release_all_pages(htex_dev);  //moved to execute case, so configuration pages are released
			htex_dev->set_done = 0;
			/*if (result != 0)
				return -ERESTARTSYS;*/
			break;
		default:
			return -ENOTTY;
	}
	return 0;
}

module_init(test_init);
module_exit(test_exit);

Re: [PCI Driver]Physical address being returned as 0 in mmap

[Kyle Moffett]

Ok, I think I understand better, further comments below.

As I asked before... please don't top-post, it is considered rude on LKML.

You should reply in-line with the other person's comments, and remove
anything not immediately relevant to the reply.

> Once the fpga receives this address it sends an interrupt to the OS to
> translate the address to a physical address which the fpga can then
> send to the dma unit. This virtual to physical address translation is
> done in the driver. So this address translation interrupt is different
> from an execution completion interrupt and there is code in place in
> the interrupt handler to distinguish these 2 types of interrupts
> properly.  The actual address translation happens in
> translate_address() defined in the device driver file. The address
> translation involves pinning the requested user page to the memory
> using get_user_pages() and then getting the physical address using
> pci_map_page(). This works because the calling process is asleep when
> this happens.

I think that there are some hardware issues here that you might not have
hit in your specific environment yet but are likely to be a problem.

First of all, you need to make sure you have all of the cache and DMA
flushes correct.  *ALL* memory that the FPGA might access while running
the user code must be cache flushed and appropriately dma_sync_*()ed.
Otherwise your FPGA will read old data.

This means in particular that you can't really read arbitrary data from the
program memory, you need to enumerate everything that the hardware
needs in advance from the kernel, or be able to flush it on demand, which
is somewhat of a performance killer.

Furthermore, many modern x86_64 systems have an "IO-MMU" which
controls access to memory individually for each device.  It essentially
provides individual page-tables for each physical device in the system
similar to the way the kernel provides virtual addressing for user programs.
If a piece of memory is not mapped in the IO-MMU then it can not be
accessed by the device at all, and even if it is mapped you need to
know the mapping address for that memory from *that* device.

Again, this boils down to the fact that you need to know in advance
(for performance reasons) exactly which user memory you want to use.


> Note that even while executing a C function which is accessing a large
> amount of data from memory, the FPGA only gets the virtual address of
> the array(in the exchange registers) and it requests a translation to
> the physical address, which is sent to the dma unit. So I decided to
> use the same process for sending the configuration data as well.

So basically what this means is that for each userspace address range
that the FPGA requests a translation for, you need to get_user_pages(),
and then dma_map_page() each page of that, and then dma_sync_*()
those pages for bidirectional access.  Unfortunately none of that can be
done from an interrupt handler (if I recall correctly) and none of it will
be fast.

Make sure you're following the locking requirements described in the
get_user_pages() documentation.


> So to improve the system what I did was not put the process to sleep
> but to let it continue. Thus i want to  continue receiving address
> translation interrupts from the fpga and then send physical addresses
> to it while the C program runs.

This is going to be a particularly bad problem.

When you are writing multi-threaded userspace programs, you MUST
ensure that you use appropriate locking operations and memory barriers,
and then your hardware will enforce cache-coherency between CPUs.

But you are effectively creating a multi-processor system *without*
full cache-coherency between your FPGA and the system CPU.  You
will need to be extremely careful to perform the cache flushes and the
DMA sync operations on the CPU in order to guarantee correctness.

And I can almost guarantee it won't be very fast.


As for the rest of it, it looks fine to me but I haven't really ever tried
to do anything that complicated.  It's already complicated enough
for me when programming a simple network controller with TX/RX
queues and such.

Good luck!

Cheers,
Kyle Moffett

Re: [PCI Driver]Physical address being returned as 0 in mmap

[newton mailinglist]

Hi Kyle,

Thanks so

> First of all, you need to make sure you have all of the cache and DMA
> flushes correct.  *ALL* memory that the FPGA might access while running
> the user code must be cache flushed and appropriately dma_sync_*()ed.
> Otherwise your FPGA will read old data.
> This means in particular that you can't really read arbitrary data from the
> program memory, you need to enumerate everything that the hardware
> needs in advance from the kernel, or be able to flush it on demand, which
> is somewhat of a performance killer.

Well yes, I actually do know exactly the data which I have to send to
the fpga in advance.
Its a fixed set of data(about 1 MB) and the CPU will not change it or
access it in any way once its inside the DMA buffer i.e. the
C program will not modify the data anymore. All I need to do is send
this fixed data to the fpga.

I was able to solve the issue finally.

Thanks so much for all the help.

So the way I do it using the DMA API is , I allocate the a buffer in
kernel space when the driver starts up :

//Allocate PCI consistent memory
cpu_addr_virt = (unsigned long)pci_alloc_consistent(htex_dev->dev,
BS_SIZE, &dma_handle);

Then whenever I need to use the buffer, I simply copy the data to it :
memset((void*)cpu_addr_virt, 0, BS_SIZE);
memcpy((void*)cpu_addr_virt, (void*)arg, BS_SIZE);

The physical addresses are known beforehand as the buffer is allocated
at startup and are valid for the entire lifetime of the driver
so no address translations are needed afterwards.

Thanks,
Abhi