Developers, Developers, Developers! Maksim Sorokin IT Blog

23May/10Off

Memory Leak in Haskell During File Read 3

Problem described in previous two posts (1 2) has finally been fully understood and solved. Guys from #haskell channel on freenode IRC servers pointed me to the problem.

The thing is that

19May/10Off

Memory Leak in Haskell During File Read 2

UPD.: Issue has been resolved

Things started to getting clearer. Here is a nice page explaining laziness during file read in Haskell (thanks to Vadim Platonov), section "hGetContents, hClose, readFile".
I have tried samples from that resource which should solve unwanted laziness, but memory leak still occurred. Moreover, those samples ran for ages on just several thousands of files and ate much more memory comparing to standard System.IO functions.

But my colleague just informed me, that the version which I posted previously runs without leaks on newer ghci version. I am too f*cked up after working day, so I hope to figure that out tomorrow.

18May/10Off

Memory Leak in Haskell During File Read

UPD.: Issue has been resolved

I used Haskell for doing certain operations on big set of XML files: parsing XML files, running regular expressions etc. Set is not that big, but still -- 90000 XML files totalling 2.7GB.
I faced memory leaks in every program that I wrote. It was quite annoying sinceĀ