flang icon indicating copy to clipboard operation
flang copied to clipboard

Introduce buffered I/O and replace getc with buffered read

Open pawosm-arm opened this issue 6 years ago • 1 comments

This commit introduces 32k buffer for file I/O operations and 32k buffer for fgetc() replacemnt routines.

This is follow-up of my work presented in pull request #642.

Some performance gain is observed and 'perf' trace does not show getc operations at all, shifting the burden to actual I/O system routines. In order to improve performance of these, 32k buffer is attached to every opened file by calling setvbuf() function.

Due to suspicious way of passing pointers around in flang runtime, all of the I/O buffers had to be allocated statically. In order to prevent memory bloat, only limited number of simultaneously opened files will be buffered, file descriptor ID is used for buffer selection which is a thread-safe solution.

The test case I was using is following:

! *********************************************************
        program main

        implicit none
!       ---------------------------------------------------
        character(len=500) :: cart
        real(kind=8) :: t1,t2

!       ---------------------------------------------------
        open(unit=9,status='old',file='my_file.txt')
        open(unit=10,file='my_new_file.txt')
        call cpu_time(t1)
        do
!               read each line
                read(9,fmt='(A)') cart
!       ************************************************************
!       ************************************************************
!                       convert  process
!       ************************************************************
!       ************************************************************

                if(cart(1:4)=='/end') then
                        write(10,*) 'this is the end!'
                        exit
                else
                        write(10,*) cart
                endif
        enddo
        call cpu_time(t2)
        close(unit=9)
        close(unit=10)

        print*,' write and read :',t2-t1




!       ---------------------------------------------------

        end program main
! *********************************************************

my_file.txt.gz

Compiled with gfortran it gives much better timing results that when compiled with flang. My patch improves the timing of flang compiled program.

pawosm-arm avatar Jan 03 '19 15:01 pawosm-arm

Anticipating your questions, setvbuf() is a very standard stdio.h function and you can find its documentation in Linux man pages and Visual Studio reference pages as well.

pawosm-arm avatar Jan 03 '19 15:01 pawosm-arm

We agreed to close this one at the last call.

bryanpkc avatar Jan 24 '24 15:01 bryanpkc