<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Developers, Developers, Developers! &#187; memory leak</title>
	<atom:link href="http://maksim.sorokin.dk/it/tag/memory-leak/feed/" rel="self" type="application/rss+xml" />
	<link>http://maksim.sorokin.dk/it</link>
	<description>Maksim Sorokin IT Blog</description>
	<lastBuildDate>Sun, 05 Feb 2012 19:37:38 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.0.3</generator>
		<item>
		<title>Memory Leak in Haskell During File Read 3</title>
		<link>http://maksim.sorokin.dk/it/2010/05/23/memory-leak-in-haskell-during-file-read-3/</link>
		<comments>http://maksim.sorokin.dk/it/2010/05/23/memory-leak-in-haskell-during-file-read-3/#comments</comments>
		<pubDate>Sun, 23 May 2010 05:13:47 +0000</pubDate>
		<dc:creator>Maksim Sorokin</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[functional programming]]></category>
		<category><![CDATA[Haskell]]></category>
		<category><![CDATA[memory leak]]></category>
		<category><![CDATA[success story]]></category>

		<guid isPermaLink="false">http://maksim.sorokin.dk/it/?p=213</guid>
		<description><![CDATA[Problem described in previous two posts (1 2) has finally been fully understood and solved. Guys from #haskell channel on freenode IRC servers pointed me to the problem. The thing is that in the memory leak example I took length of read file and appended it to accumulator, which was sent further in the tail [...]]]></description>
			<content:encoded><![CDATA[<p>Problem described in previous two posts (<a href="http://maksim.sorokin.dk/it/2010/05/18/memory-leak-in-haskell-during-file-read/">1</a> <a href="http://maksim.sorokin.dk/it/2010/05/19/memory-leak-in-haskell-during-file-read-2/">2</a>) has finally been fully understood and solved. Guys from #haskell channel on <a href="http://freenode.net/irc_servers.shtml">freenode IRC servers</a> pointed me to the problem.</p>
<p>The thing is that <span id="more-213"></span>in the memory leak <a href="http://maksim.sorokin.dk/it/2010/05/18/memory-leak-in-haskell-during-file-read/">example</a> I took length of read file and appended it to accumulator, which was sent further in the tail call. And this accumulator was evaluated lazily! After adding <a href="http://users.aber.ac.uk/afc/stricthaskell.html#seq">strict sequencing operator</a>, the memory leak gone.</p>
<p>So instead of tail call:</p>
<pre class="brush: plain; title: ;">test fns $ acc + (B.length contents)</pre>
<p>I should have used:</p>
<pre class="brush: plain; title: ;">test fns $! acc + (B.length contents)</pre>
]]></content:encoded>
			<wfw:commentRss>http://maksim.sorokin.dk/it/2010/05/23/memory-leak-in-haskell-during-file-read-3/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Memory Leak in Haskell During File Read 2</title>
		<link>http://maksim.sorokin.dk/it/2010/05/19/memory-leak-in-haskell-during-file-read-2/</link>
		<comments>http://maksim.sorokin.dk/it/2010/05/19/memory-leak-in-haskell-during-file-read-2/#comments</comments>
		<pubDate>Wed, 19 May 2010 19:15:26 +0000</pubDate>
		<dc:creator>Maksim Sorokin</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[functional programming]]></category>
		<category><![CDATA[Haskell]]></category>
		<category><![CDATA[memory leak]]></category>

		<guid isPermaLink="false">http://maksim.sorokin.dk/it/?p=193</guid>
		<description><![CDATA[UPD.: Issue has been resolved Things started to getting clearer. Here is a nice page explaining laziness during file read in Haskell (thanks to Vadim Platonov), section "hGetContents, hClose, readFile". I have tried samples from that resource which should solve unwanted laziness, but memory leak still occurred. Moreover, those samples ran for ages on just [...]]]></description>
			<content:encoded><![CDATA[<p><strong>UPD.:</strong> Issue has been <a href="http://maksim.sorokin.dk/it/2010/05/23/memory-leak-in-haskell-during-file-read-3/">resolved</a></p>
<p>Things started to getting clearer. <a href="http://users.aber.ac.uk/afc/stricthaskell.html#semiclosed">Here</a> is a nice page explaining laziness during file read in Haskell (thanks to <a href="http://lt.linkedin.com/pub/vadim-platonov/9/315/1b">Vadim Platonov</a>), section "hGetContents, hClose, readFile".<br />
I have tried samples from that resource which should solve unwanted laziness, but memory leak still occurred. Moreover, those samples ran for ages on just several thousands of files and ate much more memory comparing to standard <code>System.IO</code> functions.</p>
<p>But my colleague just informed me, that the version <a href="http://maksim.sorokin.dk/it/2010/05/18/memory-leak-in-haskell-during-file-read/">which I posted previously</a> runs without leaks on newer ghci version. I am too f*cked up after working day, so I hope to figure that out tomorrow.</p>
]]></content:encoded>
			<wfw:commentRss>http://maksim.sorokin.dk/it/2010/05/19/memory-leak-in-haskell-during-file-read-2/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Memory Leak in Haskell During File Read</title>
		<link>http://maksim.sorokin.dk/it/2010/05/18/memory-leak-in-haskell-during-file-read/</link>
		<comments>http://maksim.sorokin.dk/it/2010/05/18/memory-leak-in-haskell-during-file-read/#comments</comments>
		<pubDate>Tue, 18 May 2010 20:31:38 +0000</pubDate>
		<dc:creator>Maksim Sorokin</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[Haskell]]></category>
		<category><![CDATA[memory leak]]></category>
		<category><![CDATA[XML]]></category>

		<guid isPermaLink="false">http://maksim.sorokin.dk/it/?p=184</guid>
		<description><![CDATA[UPD.: Issue has been resolved I used Haskell for doing certain operations on big set of XML files: parsing XML files, running regular expressions etc. Set is not that big, but still -- 90000 XML files totalling 2.7GB. I faced memory leaks in every program that I wrote. It was quite annoying since  Haskell was [...]]]></description>
			<content:encoded><![CDATA[<p><b>UPD.:</b> Issue has been <a href="http://maksim.sorokin.dk/it/2010/05/23/memory-leak-in-haskell-during-file-read-3/">resolved</a></p>
<p>I used Haskell for doing certain operations on big set of XML files: parsing XML files, running regular expressions etc. Set is not that big, but still -- 90000 XML files totalling 2.7GB.<br />
I faced memory leaks in every program that I wrote. It was quite annoying since <span id="more-184"></span> Haskell was chosen as primary language for the project and I needed to produce some results relatively fast.</p>
<p>Today I decided to take closer look during what stage leak occurred. My initial assumption about leak during file read was confirmed. Initially I used <a href="http://www.zvon.org/other/haskell/Outputprelude/readFile_f.html"><code>readFile</code> to read a file. But in this example I use </a><a href="http://www.haskell.org/ghc/docs/6.12.2/html/libraries/bytestring-0.9.1.6/Data-ByteString.html"><code>Data.ByteString</code></a> since it gracefully handles file close after opening.</p>
<p>Here is my example:</p>
<pre class="brush: plain; title: ;">
module SizeAnalyzer where

import Data.ByteString as B
import Utils

docsDir = &quot;/home/mah/Documents/university/MIPH/data/updatedXml&quot;

main =
   do filenames &lt;- getFilenamesInDirectory docsDir
      acc &lt;- test filenames 0
      return acc

test [] acc = return acc
test (fn:fns) acc =
   do contents &lt;- B.readFile (docsDir++&quot;/&quot;++fn)
      test fns $ acc + (B.length contents)
</pre>
<p>Memory usage slowly grows. Even if program is finished, used memory is not freed!</p>
]]></content:encoded>
			<wfw:commentRss>http://maksim.sorokin.dk/it/2010/05/18/memory-leak-in-haskell-during-file-read/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
	</channel>
</rss>

