Using multithreading: Write file need to optimised for heavy traffic part 3 on newest questions tagged multithreading – Stack Overflow

this question is a continuation of the first 2 part, anyone who is interested to see where I come from you can refer to part 1 and part 2, but it is not necessary.

write file need to optimised for heavy traffic

Write file need to optimised for heavy traffic part 2

now i have a working snippet, the relevant part is below:

    public static class memoryStreamClass
    {
        static MemoryStream ms1 = new MemoryStream();

        public static void fillBuffer(string outputString)
        {
            byte[] outputByte = Encoding.ASCII.GetBytes(outputString);

            ms1.Write(outputByte, 0, outputByte.Length);

            if (ms1.Length > 8100)
            {
                emptyBuffer(ms1);
                ms1.SetLength(0);
                ms1.Position = 0;
            }
        }

        static void emptyBuffer(MemoryStream ms)
        {
            FileStream outStream = new FileStream("c:\\output.txt", FileMode.Append);

            ms.WriteTo(outStream);
            outStream.Flush();
            outStream.Close();
        }

the above snippet works fine, and bug free. it output around 8KB of data every write.

now i try to multithread the above code to enhance the performance of an IO write bottleneck and problems appeared. the below snippet is what i tried to attempted.

Basically i have 2 identical memoryStream, if say ms1 is full, it writes ms1 into file and switch to ms2 while ms1 is writing, and vice versa.

    public static class memoryStreamClass
    {
        static MemoryStream ms1 = new MemoryStream();
        static MemoryStream ms2 = new MemoryStream();
        static int c = 1;

        public static void fillBuffer(string outputString)
        {
            byte[] outputByte = Encoding.ASCII.GetBytes(outputString);

            if (c == 1)
            {
                ms1.Write(outputByte, 0, outputByte.Length);

                if (ms1.Length > 8100)
                {
                    c = 2;

                    Thread thread1 = new Thread( () => emptyBuffer(ms1));
                    thread1.Start();

                    ms1.SetLength(0);
                    ms1.Position = 0;
                }
            }
            else
            {
                ms2.Write(outputByte, 0, outputByte.Length);

                if (ms2.Length > 8100)
                {
                    c = 1;

                    Thread thread2 = new Thread(() => emptyBuffer(ms2));
                    thread2.Start();

                    ms2.SetLength(0);
                    ms2.Position = 0;

                }
            }
        }

the above code can compile and run, however, the output write is not always 8KB, and it writes way too frequently (than my single thread program). can someone enlighten my and points out what is wrong with my program? thank you very much

See Answers


source: http://stackoverflow.com/questions/11526856/write-file-need-to-optimised-for-heavy-traffic-part-3
Using multithreading: using-multithreading



online applications demo