mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* More filesystem corruption under 2.4.1-pre8 and SW Raid5
@ 2001-01-19 11:18 Manfred Spraul
  2001-01-19 11:36 ` Holger Kiehl
  0 siblings, 1 reply; 6+ messages in thread
From: Manfred Spraul @ 2001-01-19 11:18 UTC (permalink / raw)
  To: Holger.Kiehl, linux-kernel

> Another thing I notice is that the responsiveness of the machine
> decreases dramatically as the test progresses until it is nearly
> useless. After the test is done everything is back to normal.
> The same behavior was observed under 2.2.18.

That's expected: ext2 performs linear searches through the directory,
and with 50 000 entries that's very slow.

I'm running a few quick tests, but I don't have a large enough spare
partition (~ 1GB?) for a full test.

How much main memory do you have, how large is your raid5 partition?

Could you try to reproduce the problem with fewer files and less main
memory?

I'm running your test with 48 MB ram, 12500 files, 9 processes in a 156
MB partition (swapoff, here is the test partition ;-).
With 192MB Ram I don't see the corruption.

--
	Manfred
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

^ permalink raw reply	[flat|nested] 6+ messages in thread
* More filesystem corruption under 2.4.1-pre8 and SW Raid5
@ 2001-01-19  7:47 Holger Kiehl
  0 siblings, 0 replies; 6+ messages in thread
From: Holger Kiehl @ 2001-01-19  7:47 UTC (permalink / raw)
  To: linux-kernel

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2210 bytes --]

Hello

Trying to find a quick way to reproduce the filesystem corruption
I reported earlier, I have written a short program that simply creates
a certain number of files in a given directory. Now if I start this
program 9 times each creating 50000 files (each 2048 Bytes) in 9
different directories and then delete these files again I always get
filesystem corruption.

I admit that creating 50000 files in one directory is not something
very common, but in my other test there are simply to many process
creating and deleting files and took too long to reproduce. My
assumption is that something goes wrong somewhere as soon as a
certain number files have been created.

The test where done on two different machines both SMP, SW Raid 5
and ext2 filesystem. Under 2.4.1-pre3 and pre8 I always get filesystem
corruption. This does NOT happen under 2.2.18.

I don't know if this is due to a problem in the Raid 5, ext2 filesystem
or in the kernel. Also, I do not currently have a system with 2.4.x
without raid5. For this reason I have attached two files (one C program
and a script) with the code that corrupts my filesystem. To run it you
need to issue the following commands:

    cc -o fsd fsd.c
    mkdir testdir
    cp fsd start_fsd testdir
    cd testdir
    chmod 755 start_fsd
    ./start_fsd

    now you need to wait 3 or 4 hours and you should see some
    ext2 errors in your syslog.

WARNING: This corrupts you filesystem really badly! Sometimes
         only the files in the testdir are effected. However, I
         had cases where other files where also effected. The
         system sometimes behaves very strangely after the test,
         programs that always have worked just crash. Reconstruction
         with fsck does not always work properly, sometimes there are
         very strange files scattered over the whole filesystem
         afterwards.  So be warned, do this on a test filesystem and boot
         the machine after the test!

Another thing I notice is that the responsiveness of the machine
decreases dramatically as the test progresses until it is nearly
useless. After the test is done everything is back to normal.
The same behavior was observed under 2.2.18.

Holger

[-- Attachment #2: Type: TEXT/plain, Size: 2973 bytes --]

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <dirent.h>
#include <errno.h>

static void create_files(int, int, char *),
            delete_files(char *);


/*$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ fsd $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$*/
int
main(int argc, char *argv[])
{
   int  no_of_files,
        file_size;
   char dirname[1024];

   if (argc == 4)
   {
      no_of_files = atoi(argv[1]);
      file_size = atoi(argv[2]);
      (void)strcpy(dirname, argv[3]);
   }
   else
   {
      (void)fprintf(stderr,
                    "Usage: %s <number of files> <file size> <directory>\n",
                    argv[0]);
      exit(1);
   }
   create_files(no_of_files, file_size, dirname);
   delete_files(dirname);
   exit(0);
}


/*+++++++++++++++++++++++++++ create_files() ++++++++++++++++++++++++++++*/
static void
create_files(int no_of_files, int file_size, char *dirname)
{
   int  i, fd;
   char *ptr;

   ptr = dirname + strlen(dirname);
   *ptr++ = '/';
   for (i = 0; i < no_of_files; i++)
   {
      (void)sprintf(ptr, "this_is_dummy_file_%d", i);
      if ((fd = open(dirname, O_CREAT|O_RDWR, S_IRUSR|S_IWUSR)) == -1)
      {
         (void)fprintf(stderr, "Failed to open() %s : %s\n",
                       dirname, strerror(errno));
         exit(1);
      }
      if (lseek(fd, file_size - 1, SEEK_SET) == -1)
      {
         (void)fprintf(stderr, "Failed to lseek() %s : %s\n",
                       dirname, strerror(errno));
         exit(1);
      }
      if (write(fd, "", 1) != 1)
      {
         (void)fprintf(stderr, "Failed to write() to %s : %s\n",
                       dirname, strerror(errno));
         exit(1);
      }
      if (close(fd) == -1)
      {
         (void)fprintf(stderr, "Failed to close() %s : %s\n",
                       dirname, strerror(errno));
      }
   }
   ptr[-1] = 0;
   return;
}


/*++++++++++++++++++++++++++++ delete_files +++++++++++++++++++++++++++++*/
static void
delete_files(char *dirname)
{
   char          *ptr;
   struct dirent *dirp;
   DIR           *dp;

   ptr = dirname + strlen(dirname);
   if ((dp = opendir(dirname)) == NULL)
   {
      (void)fprintf(stderr, "Failed to opendir() %s : %s\n",
                    dirname, strerror(errno));
      exit(1);
   }
   *ptr++ = '/';
   while ((dirp = readdir(dp)) != NULL)
   {
      if (dirp->d_name[0] != '.')
      {
         (void)strcpy(ptr, dirp->d_name);
         if (unlink(dirname) == -1)
         {
            (void)fprintf(stderr, "Failed to open() %s : %s\n",
                          dirname, strerror(errno));
            exit(1);
         }
      }
   }
   ptr[-1] = 0;
   if (closedir(dp) == -1)
   {
      (void)fprintf(stderr, "Failed to closedir() %s : %s\n",
                    dirname, strerror(errno));
   }
   return;
}

[-- Attachment #3: Type: TEXT/PLAIN, Size: 277 bytes --]

#!/bin/sh

NO_OF_PROCESS=9
NUMBER_OF_FILES=50000
FILE_SIZE=2048

counter=0
while [ $counter -lt $NO_OF_PROCESS ]
do
   if [ ! -d $counter ]
   then
      mkdir $counter
   fi
   ./fsd $NUMBER_OF_FILES $FILE_SIZE $counter &
   counter=`expr "$counter" + 1`
done

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2001-01-19 12:49 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-01-19 11:18 More filesystem corruption under 2.4.1-pre8 and SW Raid5 Manfred Spraul
2001-01-19 11:36 ` Holger Kiehl
2001-01-19 11:59   ` Manfred Spraul
2001-01-19 12:49     ` Holger Kiehl
2001-01-19 12:02   ` Nils Rennebarth
  -- strict thread matches above, loose matches on Subject: below --
2001-01-19  7:47 Holger Kiehl

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®