Hi Guys,
I have actually done some work around this in the past using Director, Python, Unix2Dos, AWK, Perl and sed to find the quickest approach and using the Director Search and Replace task is just as quick as any other methods.
For all of my tests I used a 51MB CSV file that contained 2,823,001 lines,
Director
In Director you can just use a Search and Replace task
Code: Select all<project name="convertlf" mainModule="Main" version="2.0">
<module name="Main">
<searchAndReplace inputFile="/root/testFiles/test.csv" outputFile="/root/testFiles/converted.csv" searchFor="\n" replaceWith="\r\n" version="1.0" disabled="false" />
</module>
</project>
Code: Select all18/10/13 12:22:47.0492 PM INFO Executing task 'searchAndReplace 1.0'
18/10/13 12:22:48.0865 PM INFO 1 file(s) were scanned for '\n' and a total of 2,823,000 replacements were made with '\r\n'
18/10/13 12:22:48.0865 PM INFO Finished task 'searchAndReplace 1.0'
Unix2Dos
You can call Unix2Dos as a native command and all you need to pass is the input file but for my test I did not run it in Director.
Code: Select all[email protected]:~/testFiles# time unix2dos test.csv
unix2dos: converting file test.csv to DOS format ...
real 0m1.065s
user 0m0.932s
sys 0m0.112s
Python
Code: Select alldata = open("test.csv", "rb").read()
newdata = data.replace("\n","\r\n")
if newdata != data:
f = open("test.csv", "wb")
f.write(newdata)
f.close()
Perl
Code: Select all[email protected]:~/testFiles# time perl -p -e 's/\n/\r\n/' < test.csv > converted.csv
real 0m3.055s
user 0m2.868s
sys 0m0.136s
These 2 were more for fun than anything else and only apply if you are running on a Unix / Linux system
AWK
Code: Select all[email protected]:~/testFiles# time awk 'sub("$", "\r")' test.csv > converted.csv
real 0m0.631s
user 0m0.436s
sys 0m0.192s
sed
Code: Select all[email protected]:~/testFiles# time sed 's/$'"/`echo \\\r`/" test.csv > converted.csv
real 0m3.436s
user 0m3.312s
sys 0m0.096s
