Results 1 to 4 of 4

Thread: Does java parse large text file

  1. #1
    Join Date
    Nov 2008
    Posts
    94

    Does java parse large text file

    I have fairly large text file in Java and the size for that file would be nearly about 20 mb. Could i be able to use the java for doing this or should i go for shell scripting? and what could be the maximum size of a file that could be parsed with java. Like for example can i parse a txt file whose size is > 20 m.b. and parsing such big file will have any impact on the perfomance.

  2. #2
    Join Date
    Mar 2008
    Posts
    258

    Re: Does java parse large text file

    Java does not have a built in method to count the number of times a String appears. Fortunately, it's quite simple to implement one. garbage collection is something to keep in mind. If you produce hundreds of thousands of tokens from the file you're parsing, then you will need to tune the VM for max performance.

  3. #3
    Join Date
    Apr 2008
    Posts
    117

    Re: Does java parse large text file

    The parsing is some text files that are stored in a folder and the size of the each file could be between 20 to 30 m.b ColdFusion parses XML document and it needs to able to load and parse the entire document in memory before it can return a result to you. I wouldn't dare say that without some figures to back it up. Both Scanner and String.split use a java.util.regex.Pattern object.

  4. #4
    Join Date
    Nov 2008
    Posts
    94

    Re: Does java parse large text file

    I am using the substring method of the String class to parse out the individual fields. I read the input record into a String variable and substring out the individual fields. Some of the files have as many as 150-200 fields on a record. I have also notice that parsing is done for small file, but while passing huge size file , the JVM memory issue occurs

Similar Threads

  1. How to parse comma separated text into Multiple Fields?
    By Leonard in forum Windows Software
    Replies: 5
    Last Post: 12-03-2010, 05:39 AM
  2. How to add parse in scanner in java?
    By MAGAR in forum Software Development
    Replies: 5
    Last Post: 02-03-2010, 03:30 PM
  3. Powershell script to parse system logs in text file
    By SADIQ in forum Operating Systems
    Replies: 2
    Last Post: 02-06-2009, 07:05 PM
  4. Batch Script Text file parse
    By tator.usenet@gmail.com in forum Windows Server Help
    Replies: 5
    Last Post: 25-03-2009, 02:12 AM
  5. Batch Script to parse lines in text file
    By jntoner1@gmail.com in forum Windows Server Help
    Replies: 8
    Last Post: 28-01-2009, 03:38 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •