vmedrv

- VME device driver for Linux 2.x & 3.x -

Japanese

'vmedrv' is a device driver for Linux 2.x for the PCI-VME bus adapter Model 616/617/618/620 manufactured by SBS Technologies (Bit3). The vmedrv driver is a loadable module with full VME interface functionality, such as programmed I/O, memory mapped access (mmap()), VME interrupt handling, and DMA data transfer.

This driver was written as carefully as possible, and is distributed in the hope that it will be useful, but it comes with ABSOLUTELY NO WARRANTY. Please do not use it if you do not agree with these terms. The licence for this software is set as "LGPL Version 2.1". For details of LGPL2.1, please refer to COPYING-LGPL2.1 file inside the distributed packege.

The vmedrv device driver was developped as a part of the Kinoko project, which aims to develop a general-purpose, scalable and extensible data acquisition system. For further details about the Kinoko project, please refer to the KiNOKO Home Page.


Available Platforms

The Linux version and implemented functions as of now are shown in the following chart.

OS PIO Mapped I/O Interrupt
Handling
DMA Support Models
Linux 2.0.x only read() available available only read() SBS 617
Linux 2.2.x available available available available SBS 617/618/620
Linux 2.4.x available available available available SBS 616/617/618/620
Linux 2.6.x
Linux 3.x
available available available available SBS 616/617/618/620

The following is a list of verified Linux distributions.

kernel 3.x

Ubuntu 11.10, 12.04 LTS (32bit / 64bit)
No change is needed. Use vmedrv for Linux 2.6.
g++ package must be installed.

Fedora 16 (32bit / 64bit)
No change is needed. Use vmedrv for Linux 2.6.
gcc-c++ package must be installed.
appropreate kernel-devel package must be installed. See example below:
% sudo yum install kernel-devel-`uname -r`

kernel 2.6

Debian 6.0 Squeeze, X86_64 (64bit)
No change is needed.

Fedora 8 / 10 / 12
No change is needed.
Ubuntu 10.04 LTS
No change is needed.

Fedora Core 6
The 'kernel-devel' package must be installed before compiling the driver.
Fedora Core 5
The 'kernel-devel' package must be installed before compiling the driver.
Fedora Core 4
No change is needed.
Fedora Core 3
Set the SYSTEM variable in the Makefile to "FC3".
Fedora Core 2
Set the SYSTEM variable in the Makefile to "FC2".

kernel 2.4

Fedora Core 1
The 'kernel-source' package must be installed before compiling the driver.
Set the KERNEL_INCLUDE_DIR variable in the Makefile to "/usr/src/linux-2.4/include".
Red Hat Linux 8.0
The 'kernel-source' package must be installed before compiling the driver.
Set the KERNEL_INCLUDE_DIR variable in the Makefile to "/usr/src/linux-2.4/include".
Set the SYSTEM variable in the Makefile to REDHAT.
Vine Linux 2.5
Set the KERNEL_INCLUDE_DIR variable in the Makefile to "/usr/include".
Red Hat Linux 7.1J
The 'kernel-source' package must be installed before compiling the driver.
Set the KERNEL_INCLUDE_DIR variable in the Makefile to "/usr/src/linux-2.4/include".

kernel 2.2

Red Hat Linux 7.0J
The 'kernel-source' package must be installed before compiling the driver.
Set the KERNEL_INCLUDE_DIR variable in the Makefile to "/usr/src/linux/include".
Red Had Linux 6.2 / 6.2J
No change is needed.
Kondara MNU/Linux 2000
No change is needed.
Vine Linux 2.1.5
No change is needed.
Vine Linux 2.0
No change is needed.
Debian GNU/Linux (woody)
No change is needed, but the 'kernel-headers' package might need to be installed.
Debian GNU/Linux (potato)
No change is needed, but the 'kernel-headers' package might need to be installed.

Download and Feedback

If you have any questions, comments, etc, please email the system developer at http://www.awa.tohoku.ac.jp/~sanshiro/kinoko-feedback/sendmail-e.html

Hardware Configuration

Factory settings for the SBS Model-617/618/620 are as follows.

So if there is already a controller and the user does not try to obtain interrupts, it can be used right away.

To use the adaptor as a VME controller, the following jumper settings in the SYS block is needed.

To use the adaptor as the controller, the following jumper settings must be set in the BGI-BGO to make the adaptor function as a bus arbiter.

To obtain VME interrupts through the interface cards, the following jumpers under I-INT block must be set in order to transmit the VME interrupt to the PC.

In order to use the adapter as a controller, the adapter must be in the far left slot. Also, unless the crate being used is a fairly sophisticated type, all backplane jumpers (BG[0-3]-IN/BG[0-3]-OUT, IACK-IN/IACK-OUT) that correspond to slots in the crate that does not have any module in it must be connected. So in other words, do remember to disconnect jumpers IACK-IN/IACK-OUT behind the modules that are used to request interrupts.

Installation

  1. Before installing the driver, connect the VME adaptor to the PC and make sure the VME power is on.

  2. Unpack the tarfile, and go to the directry with the driver source.
        % gunzip vmedrv-1.0.0.tar.gz
        % tar xvf vmedrv-1.0.0.tar
        % cd vmedrv/Linux2.6_Bit3_617
    

    The versions of Linux and vmedrv should be changed to the versions you are using. In the vmedrv for Linux2.2/2.4, Model 618/620 uses the same code as 617. Model 618/620 cannot be used as the vmedrv for Linux2.0.

  3. Edit Makefile according to your environment. Change the following variables near the begining in Makefile. The environments the developer uses have settings listed in the "Avaliable Platform" section so please use those as a reference.

    KERNEL_INCLUDE_DIR
    The directry that has the kernel header files. Must be exactly the same as the header files used in compiling the kernel currently running.

    In some distributions, the kernel header in the "kernel-headers" package is not the one that was used when compiling. For these distributions, install the kernel-source package first and then indicate the header files inside the package (you do not need to compile the kernel source).

    If you built your own kernel, indicate the header files' directory of the source you used.

    USE_MODVERSIONS
    If the kernel running was compiled with MODVERSIONS enabled, choose 1, if not, choose 0. In most recent distributions, MODVERSIONS seems to be used (indicate "1" ).

    If you rebuilt the kernel, indicate according to the settings you chose.

    To find out if MODVERSIONS is used in your current kernel, "cat" in /proc/ksyms and see if the kernel function has the version information added.
    % cat /proc/ksyms | grep kmalloc
    c012e880 kmalloc_R93d4cfe6            <-- MODVERSION used
    % cat /proc/ksyms | grep kmalloc
    c012e880 kmalloc                      <-- MODVERSION not used
    

    SYSTEM (only for Linux 2.4)
    If you are using RedHat Linux, set SYSTEM=REDHAT, if not set it as OTHERS. This is to accomodate for the changes in kernel that the RedHat system made. Especially in RedHat 9 this is important.

    If the kernel source was brought in from outside of RedHat, set SYSTEM=OTHERS even if you do use RedHat.

  4. Compile the driver.
        % make
    

  5. Switch user to root and install the driver. Use the 'dmesg' UNIX command to check the installation status.
        % su
        # make install
        # dmesg
               [many many lines here]
        vmedrv: Bit3 617 Bus Adapter was detected at ioport 0xdcc0 on irq 10.
          I/O Mapped Node at 0xdcc0.
          Memory Mapped Node at 0xfe010000.
          Mapping Register at 0xfe000000.
          Remote Memory at 0xf8000000.
          Adapter ID (I/O): 0x80
          Adapter ID (mem): 0x80
        vmedrv: successfully installed at 0xdcc0 on irq 10 (major = 127).
        #
    

  6. Everytime the Linux system is rebooted, the driver must be installed again.
        # make install
    

    Refer to Kinoko-DAQ technical tips for information on loading the driver automatically when rebooting.

  7. To remove the driver from the operating system, use the following commands:
        # make uninstall
        # dmesg
               (many many lines here)
        vmedrv: removed.
        #
    

If a user is using a newer version of Linux than the one the developer used, there is a possibility that compiling and installing will end unsuccessfully. In this case, please let us know by sending an e-mail to the system developer at http://www.awa.tohoku.ac.jp/~sanshiro/kinoko-feedback/sendmail-e.html

Testing the Driver

The vmedrv driver package comes with some sample/test programs. These programs can be used to verify the driver installation. You may have to edit the test programs according to your own hardware configuration before compiling them.
% cd ..
% edit the test programs
% make
The following is a list of the test programs. If they are executed without any arguments passed, a simple usage will be displayed.
vmeread.c / vmewrite.c Test program for programmed I/O
vmemap.c Test program for memory mapped I/O
vmeint.c / vmeintwait.c Test program for interrupt handling
vmedmaread.c / vmedmawrite.c Test program for DMA transfer

To execute vmeread.c, vmewrite.c, vmemap.c, vmedmaread.c, or vmedmawrite.c, you need to install a VME memory module (or something that behaves similarly) into your VME crate. These test programs use A32D32 transfer mode as the default. You can change the transfer mode by editing the following line in the begining of each source file:

    #define DEV_FILE "/dev/vmedrv32d32"

To execute vmeint.c, you need to install a VME module that issues VME interrupts. Since the test program does not include code to configure the interrupt registers on the module, you might have to add some code to do that. In the test program, codes to configure interrupt registers on RPV-130 (Rinei) are written in a comment block as an example. The default transfer mode of this test program is A16D16.

Usage

vmedrv is very useful when used through KiNOKO, a general purpouse data acquisition system. For details, please see the KiNOKO Homepage.

vmedrv is most useful when used through KiNOKO, but the way to directly access the driver is written below. Also, programs used to test the driver will be very useful as a sample.

Open the Driver
Use the system call open() to open the driver.
    int fd = open("/dev/vmedrv32d32", O_RDWR);
When the vmedrv driver is installed, the following entries will be created in the /dev directory. The drivers differ by their default transfer modes. The transfer mode can be changed at any time by calling ioctl().

Entry File Name Default Transfer Mode
/dev/vmedrv not explicitly specified
/dev/vmedrv16d16 Address 16bit, Data 16bit, PIO
/dev/vmedrv16d32 Address 16bit, Data 32bit, PIO
/dev/vmedrv24d16 Address 24bit, Data 16bit, PIO
/dev/vmedrv24d32 Address 24bit, Data 32bit, PIO
/dev/vmedrv32d16 Address 32bit, Data 16bit, PIO
/dev/vmedrv32d32 Address 32bit, Data 32bit, PIO
/dev/vmedrv24d16dma Address 24bit, Data 16bit, DMA
/dev/vmedrv24d32dma Address 24bit, Data 32bit, DMA
/dev/vmedrv32d16dma Address 32bit, Data 16bit, DMA
/dev/vmedrv32d32dma Address 32bit, Data 32bit, DMA

Even if the transfer mode is set to DMA, PIO will be used for memory mapped I/O.

The returned value is the new file descriptor to the driver. If an error occures, a negative number will be return and the global variable errno will be set appropriately. For further information, refer to the man page of open(2).

Seek
Use the system call lseek() to set the current VME (offset) address.
    lseek(fd, offset, SEEK_SET);
offset is the VME address. Values that can be used for the third argument are SEEK_SET and SEEK_CUR only. SEEK_END cannot be used here. The return value is the new offset address. If an error occurs, a negative number will be returned and the global variable errno will be set appropriately. For further information, refer to the man page of lseek(2).

PIO Read
Use the system call read().
    int size = read(fd, buffer, max_size);
The number of bytes read is returned, and the current position is advanced by this number. In the case of an error, a negative number is returned and the global variable errno is set appropriately. For further information, refer to the man page of read(2).

PIO Write
Use the system call write().
int size = write(fd, buffer, size);
The number of bytes written is returned. In the case of an error, a negative number is returned and the global variable errno is set appropriately. For further information, refer to the man page of write(2).

Memory Mapped I/O
Use the system call mmap().
    void* start = mmap(0, length, prot, flags, fd, offset);
The value offset is the VME address, and length is the size of mapping. "offset" must be an integer multiple of MMU pagesize. If you want to map from an address that is not an integer multiple, round down the start address to pagesize and add the appropriate size later to the returned address.

In CPUs of x86 series after 386, MMU pagesize is 4kB. For other CPUs, refer to the PAGE_SIZE definition (probably macro) in $(KERNEL_INCLUDE_DIR)/asm/page.h.

The return value is the start address of the mapped memory. If an error occurs, a negative number is returned and the global variable errno is set appropriately. For further information, refer to the man page of mmap(2).

Setting Interrupt Configurations and Handling
Use the system call ioctl() to set interrput configurations.
    struct vmedrv_interrupt_property_t interrupt_property;    
    interrupt_property.irq = IRQ;
    interrupt_property.vector = VECTOR;
    interrupt_property.signal_id = SIGNAL_ID;

    /* register interrupt */
    ioctl(fd, VMEDRV_IOC_REGISTER_INTERRUPT, &interrupt_property);

    /* enable interrupt */
    ioctl(fd, VMEDRV_IOC_ENABLE_INTERRUPT);
    
    /* the interrupt is processed here */
    /* the registered SignalID due to the VME interrupt is issued */

    /* disable interrupt */
    ioctl(fd, VMEDRV_IOC_DISABLE_INTERRUPT);

    /* unregister interrupt */
    ioctl(fd, VMEDRV_IOC_UNREGISTER_INTERRUPT, &interrupt_property);
Here the interrupt vector the vmedrv uses is NOT 8bits but 16 bits. For modules that return 8 bit vectors, fill the top 8 bits of the vector that is specified in ioctl() with 1 (ex. 0xf0 -> 0xfff0).

By enabling interrupts, interupt handling will start for all registered interrupts. If a registered interrupt occurs, the signal registered with the interrupt will be raised.

Waiting For Interrupts
Set SignalID to 0 and specify VMEDRV_IOC_WAIT_FOR_INTERRUPT in ioctl().
struct vmedrv_interrupt_property_t interrupt_property;    
interrupt_property.irq = IRQ;
interrupt_property.vector = VECTOR;
interrupt_property.signal_id = 0;

/* register interrupt */
ioctl(fd, VMEDRV_IOC_REGISTER_INTERRUPT, &interrupt_property);

/* enable interrupt */
ioctl(fd, VMEDRV_IOC_ENABLE_INTERRUPT);

/* wait for interrupt */
interrupt_property.timeout = TIMEOUT_SEC;
result = ioctl(fd, VMEDRV_IOC_WAIT_FOR_INTERRUPT, &interrupt_property);
if (result > 0) {
    /* obtain VME interrupt */
}
else if (errno == ETIMEDOUT) {
    /* timeout */
}
else if (errno == EINTR) {
    /* interrupted by other signals, etc */
}
else {
    /* error */
}

/* disable interrupt */
ioctl(fd, VMEDRV_IOC_DISNABLE_INTERRUPT);

/* unregister interrupt */
ioctl(fd, VMEDRV_IOC_UNREGISTER_INTERRUPT, &interrupt_property);
Here the interrupt vector the vmedrv uses is NOT 8bits but 16 bits. For modules that return 8 bit vectors, fill the top 8 bits of the vector that is specified in ioctl() with 1 (ex. 0xf0 -> 0xfff0).

DMA Read
Use the system call ioctl() to set the transfer mode to DMA, then use read() to read from the current VME address.
    /* Set the transfer mode to DMA. */
    /* If the default mode is DMA, the line below can be omitted. */
    int transfer_method = VMEDRV_DMA;
    ioctl(fd, VMEDRV_IOC_SET_TRANSFER_METHOD, &transfer_method);

    int size = read(fd, buffer, max_size);
If successful, the number of bytes read is returned.

During DMA transfer, do not access the VME. Also, all VME interrupt handling processes are postponed until DMA transfer is completed.

On error, a negative number is returned and the global variable errno is set appropriately. For further information, refer to the man page of read(2).

DMA Write
Use the system call ioctl() to set the transfer mode to DMA, then use write() to write-in.
    /* Set the transfer mode to DMA. */
    /* If the default mode is DMA, the line below can be omitted. */
    int transfer_method = VMEDRV_DMA;
    ioctl(fd, VMEDRV_IOC_SET_TRANSFER_METHOD, &transfer_method);

    int size = write(fd, buffer, size);
If successful, the number of bytes read is returned.

During DMA transfer, do not access the VME. Also, all VME interrupt handling processes are postponed until DMA transfer is completed.

On error, a negative number is returned and the global variable errno is set appropriately. For further information, refer to the man page of write(2).

Close the Driver
Use close() system call to close the driver. Call munmap() system call in advance if the VME memory is mapped with mmap(). For further information, refer to the man page of close(2) and munmap(2).
    munmap(start, length);
    close(fd);

Known Bugs


Edited by: Enomoto Sanshiro