Convert Unix line endings to Windows
I recently moved back to Windows from Linux. I have some files with CRLFs, some with LFs and some that are mixed. Is there a utility that will help me find all my Unix-touched files and convert them to proper CRLF terminated files? The utility must run on Windows, not Linux. I have already moved. I’d rather not install Cygwin if I can avoid it.
related: see superuser.com/questions/38744 superuser.com/questions/27060 superuser.com/questions/52044 . any tool mentioned in those can be reversed to accomplish what you want to do
The solutions in those so called Duplicate questions all run on linux platforms or preform the opposite conversion. I don’t have linux any more. I have windows.
they are all available for Windows via Cygwin, GnuWin32, UnxUtils or the like. i believe there are Powershell scripts that can do this as well, though i don’t know enough about it to provide any links.
@quack: The question specifically states that the user is looking for something that will do the job on Windows, using a Windows utility.
14 Answers 14
You can convert them with the unix2dos utility on your Linux platform. There are unix2dos versions available for Windows as well.
If you have Perl installed you can also use this one liner:
perl -p -e 's/\n/\r\n/' < UNIX_LF.txt >WINDOWS_CRLF.txt yes and no. I WROTE my own blasted utility. I could have done this from the start, but I was hoping to save 30 mins. Unfortunately I was not happy with any utilities available. The closest I found corrupted any binary files in the directory.
There are problems converting with dos2unix and unix2dos — if the file has mixed CRLF and LF there may be some occurences that wouldn’t be replaced.
Here is an easy and quick way.
Drag and drop the text file into Chrome (I don’t know about other browsers) and then cut and paste back into the original file 🙂
all browsers can read Unix line endings just fine, including IE. But even then wordpad can also do that, and so will modern Notepad. What’s important here is how to convert multiple files automatically
The one I found best for recursively going through folders, allowing file filters and allowing a simple search for «\r\n» and replacing it with just «\n» was Notepad++.
Notepad++ is one of the best, free, open source notepad programs for Windows. It is very simple and powerful. It handled the line ending search/replace just fine. A contractor check a bunch of .c and .h files in to our repository with Linux \r\n line endings, but since most people have standardized on Windows/Eclipse build tools, the files won’t build until the line endings are converted.
For example: sfk addcr -dir . -file .txt -norec
changes LF endings into CR/LF for Windows, on all .txt files of the current directory, but NOT within subdirectories (no recursion).
But this program does a lot more than just that.
On Cygwin, you can convert between Unix and «DOS» AKA Windows files using two built-in utilities:
Convert to DOS CR/LF format:
Convert back to Unix CR format:
The file is left in place with the same name.
This is sometimes called dos2unix or unix2dos in case you cannot find it as u2d or d2u. There is also unix2mac
I’m going to throw this solution out there. Git will do this. See this post about it
So theoretically you could do this to convert an entire tree
cd root/of/tree git init . git add . git commit -m "initial commit" echo "* text eol=crlf" > .gitattributes git rm --cached -r . git reset --hard Change crlf to lf if you want to go the other way. NOTE: you’re not done yet, keep reading
Type git status to see which files will be affected. You might have to add lines like
*.jpg binary *.png binary *.gif binary etc to .gitattributes to avoid converting certain files. You can also explicit mark certain files as text
Then just repeat these 2 lines after you’ve edited .gitattributes
git rm --cached -r . git reset --hard Then use git status again to see which files will be changed. When you’re sure all the files you want affected are listed by git status then commit
git add . git commit -m "normalize line endings" now check all the files out again
git rm --cached -r . git reset --hard They should now have whatever your desired line endings are
** NOTE: If you were already using git skip the first 3 commands git commands. If you were not using git you can now delete the .gitattributes file and the .git folder.
** Back up your files: the git rm —cached -r deletes them all (although they are theoretically in your git repo (the .git folder) which is how they get restored by the last command git reset —hard . It’s just since files are getting deleted it’s probably best to back them up.
TYPE unix_file | FIND /V "" > dos_file In PowerShell there are various ways to do that by reversing what was done in this question.
PS> (Get-Content $file -Raw).Replace("`n", "`r`n") | Set-Content $path -Force PS> (Get-Content $infile) -join "`r`n" > $outfile PS> $text = [IO.File]::ReadAllText($original_file) -replace "`n", "`r`n" PS> [IO.File]::WriteAllText($original_file, $text) It’s also possible to do this in VBScript and JScript which are also tools that are already available in Windows without installing a third-party application.
Use a text editor that understands both line endings like SciTE, or Notepad++ if you don’t need to convert all line ending in all your files, but just don’t want to see the entire file bunched up on the first line.
That’s what they said when I had the inverse problem while switching to linux. I kept running into tools that had problems with the wrong line endings. And NOTHING handles mixed line endings reasonably.
@Matthew I know the scite handle unix / windows line endings correctly. I use it for that purpose in the windows vms I run in linux when editing files on the host system.
I am sure that the editors you recommend will work just fine. but at some point, I will need to feed the textfile to some other tool, and that tool will NOT. At that point, having a text editor that handles things well will make it worse by hiding the problem. and I will bet that it still handles mixed endings badly (or at least with quirks).
+1 for the notepad++ option to convert lie endings. And here is how to do more specifically (see «method 2») : staffwww.fullcoll.edu/brippe/csci123/saveAsUnix.aspx
There are many ways to translate the eoln characters in TEXT files, and everyone has a favourite.
But I always transfer files from Linux to Windows in BINARY mode, then I open TEXT files in Windows with an editor capable of opening both types, and saving them in either form, if necessary.
I used Programmers File Editor for this, but Notepad++ can do it too. WordPad is also useful [at least, for viewing LF terminated files].
I’m thinking of the simple text files which may have originated on Linux, and need to be readable in the (defacto standard) world of Windows. I’m not sure what you meant by ‘unix-touched files’.
Convert line endings [duplicate]
I have been using d2u to convert line endings. After installing Puppy Linux I noticed that it does not come with d2u , but dos2unix . Then I noticed that Ubuntu is missing both by default. What is another way to convert line endings?
2 Answers 2
Using tr
Using perl
Using sed
sed 's/^M$//' windows.txt > unix.txt sed 's/\r$//' windows.txt > unix.txt To obtain ^M , you have to type CTRL-V and then CTRL-M .
I learned on mac you cannot use tr to open and write to the same file. That results in a blank file, but writing to a different name works great!
@Loren i think that should be your assumption with any redirection. The destination file is opened before the reading of the source. Some commands let you do «in-place» like sed ‘s -i but use intermediate/backup files anyway
These answers are generally correct and you could add awk ‘
Redirecting stdout and stdin in the same line kinda messes with my head so I used: cat windows_newlines.txt | tr -d ‘\r’ > unix_newlines.txt
Doing this with POSIX is tricky:
- POSIX Sed does not support \r or \15 . Even if it did, the in place option -i is not POSIX
- POSIX Awk does support \r and \15 , however the -i inplace option is not POSIX
- d2u and dos2unix are not POSIX utilities, but ex is
- POSIX ex does not support \r , \15 , \n or \12
To remove carriage returns:
Those awk scripts are GNU awk only due to multi-character RS (more than 1 char in a RS invokes undefined behavior in POSIX so some POSIX awks will silently drop the $ and retain just the ^ , others can do whatever else they like), they would produce unexpected results when the getline fails, they will only operate on the first line of the input, and they will corrupt the input file in some situations and if they were fixed to operate on all lines would cause an infinite loop in others by writing to the input file as it’s being read. Do not execute those scripts.
This work on the same file, ie. it replace line endings in-place. While the tr solutions require different file as an output.
How to change line-ending settings
I actually find that the 3-rd option works better. Otherwise I often have been in situations when I edit both batch and sh scripts on the same platform (Windows/Linux) and then commit them and Git automatically «fixes» line endings for one platform. No, I prefer to be self-conscious about line endings and commit/checkout them exactly as they are.
Agree with @JustAMartin having the system messing with your line endings is a great way to introduce bugs that will take an entire day to track down and fix, as I just have. All decent editors and IDEs on Windows now fully support LF line endings nowadays, there is no need for this translation.
@Neutrino I wish this was true, but one example of IDE that messes with your line endings (and doesn’t offer a reasonable configuration option to turn this off) is Visual Studio.
8 Answers 8
The normal way to control this is with git config
git config --global core.autocrlf true For details, scroll down in this link to Pro Git to the section named «core.autocrlf»
If you want to know what file this is saved in, you can run the command:
and the git global config file should open in a text editor, and you can see where that file was loaded from.
Actually, if you re-read your own question, in the copy/pasted excerpts : «1 . («core.autocrlf» is set to «true») . 2 . («core.autocrlf» is set to «input») . 3 . («core.autocrlf» is set to «false»)» so you basically answered your own question? 🙂
Pro git has been absorbed by git-scm. An equivalent link to the git-scm book is the formatting and whitespace configuration section.
Line ending format used in OS:
We can configure git to auto-correct line ending formats for each OS in two ways.
Global Configuration
In Linux/OSX
git config --global core.autocrlf input This will fix any CRLF to LF when you commit.
In Windows
git config --global core.autocrlf true This will make sure that, when you checkout in windows, all LF will be converted to CRLF .
.gitattributes File
It is a good idea to keep a .gitattributes file as we don’t want to expect everyone in our team to set their own config. This file should be placed in the repository root and. If it exists, git will respect it.
This will treat all files as text files and convert to OS’s line ending on checkout and back to LF on commit automatically. If you want to specify the line ending explicitly, you can use:
The first one is for checkout and the second one is for commit.
This will treat all .jpg images as binary files, regardless of path. So no conversion needed.
Or you can add path qualifiers:
According to gitattributes documentation setting * text=auto lets git decide whether the content is text or not. Forcing all files to be text should be * text only.
The first one is for checkout and the second one is for commit. Are you sure? I see no documentation supporting this
For a repository setting solution, that can be redistributed to all developers, check out the text attribute in the .gitattributes file. This way, developers dont have to manually set their own line endings on the repository, and because different repositories can have different line ending styles, global core.autocrlf is not the best, at least in my opinion.
For example unsetting this attribute on a given path [. — text] will force git not to touch line endings when checking in and checking out. In my opinion, this is the best behavior, as most modern text editors can handle both type of line endings. Also, if you as a developer still want to do line ending conversion when checking in, you can still set the path to match certain files or set the eol attribute (in .gitattributes) on your repository.
Also check out this related post, which describes .gitattributes file and text attribute in more detail: What’s the best CRLF (carriage return, line feed) handling strategy with Git?