How to work with large tar files on linux

[Harold Elwin]

Sponsored Links

I am endeavoring to back up my Linux box to my windows box's hard drive. To do this I am utilizing the Knoppix distro to boot my Linux box. Then I am tarring and ftping each file and sending it to my windows box through ftp. (I required to tar the files primary, so I be able to preserve permissions) On my windows xp box I am working with filezilla's ftp server, and I am transferring to an exterior 320Gb NTFS formatted hard drive attached to it commencing from side to side usb. I do not have sufficient space left on my Linux box to tar the whole thing and then transfer, so I am utilizing the following commands ftp 192.168.1.101 21 binary position |"tar -cvlO stuff.tar It forever stops transferring immediately previous to 2Gb (1,972,460KB), and the file be supposed to be 20Gb or so. What am I doing wrong? Is there some file size limit by means of the intention of I do not be familiar by means of of for ftp or tar. I need some help here.

[Zhankana_n]

Subsequent to googling approximately for hours I believe the tribulations by means of a number of my tar files demonstrating a 403 prohibited message at the same time as attempting to utilize WGET is for the reason by means of the intention of of the large file size. I encompass 100GB of data to transfer as an important person purchased my 3 year old social network site. I have comprised decided to tar each directory independently and am left by means of regarding 4 directories by means of the intention of are approximately connecting 9GB and 40GB each. Is there a method to span tar files so by means of the intention of I be able to generate a tar file additional than have them split into multiple files <2GB in case the size is the tribulations. So the query remains, is there a moderately quick technique to transfer large TAR files commencing from server to server.

[teena_pansare]

They are more often than not premeditated to do scheduled backups to a server or hard disc additional than I required a straightforward method to create one-off backups of a number of large files to DVDs or a USB drive. I determined, therefore, to utilize an uncomplicated command line procedure to do the majority of the work. It takes merely two commands to generate the archive and three to restore the archive and make sure it for errors. The attractiveness of the command line approach is by means of the intention of, on one occasion you have a working procedure, and you be able to save the commands in a text file (called a shell script) and connection it to an icon on your desktop. Anytime you necessitate to repeat the procedure, basically hit it off the icon. The shell script in addition serves to document the procedure and you are able to easily edit it if you necessitate to change it.

[Ekpah]

I have comprised a directory containing a number of files for a virtual system. The files by means of the intention of hold the data for the virtual systems hard discs are not merely big, they are in addition "sparse" files, which means by means of the intention of they utilize merely sufficient disc space for the data by means of the intention of was essentially written to the file. For illustration, the virtual system might have a 30GB drive of which 2GB has been utilized. Even though it uses merely 2GB on my hard disc, a program by means of the intention of reads the file might observe it as a 30GB file. This type of file be able to be tricky to back up for the reason by means of the intention of, when you copy it, you be able to end up by means of a 30GB file, or it might basically fail to copy, depending on the type of file system utilized on the backup storage.

[windows_user]

A simple method to generate an archive by means of the intention of uses no additional space than is necessary for sparse files, as well as protect links, files ownerships and permissions, is to utilize the venerable tar program. This packs the files into a solitary "archive" file and can compress the files as well. Imagine of it as the Unix correspondent of a ZIP file. The command would give the impression of being incredible approximating tar to be archived and myarchive.tgz is the name of the collection file to be generated. I desire the archive to be straightforward to copy to a variety of storage media similar to a USB flash drive and dissimilar formats of DVD. These encompass limits on the maximum size of a file which be able to be as diminutive as 1GB so I utilize the split program to divide the documents into 1000MB chunks.

[Pomegranate]

The tar archive format obliges an upper frontier of 68GB on the size of several solitary file by means of the intention of be able to be added to a tar archive. I believe the GNU version or description of the tar program determine to handle files up to this size additional than some other versions or descriptions might encompass a limit of 8GB. If you have comprised files larger than the maximum then, as long as they are not sparse files, you be able to overcome the size restriction by compressing and/or splitting these files previous to creating the archive (e.g. utilizing the gzip and split commands). As far as I be familiar by means of, there is no upper limit on the total size of the tar archive or the number of files it might contain.