Go Back   TechArena Community > Software > Software Development
Become a Member!
Forgot your username/password?
Register Tags Active Topics RSS Search Mark Forums Read

Sponsored Links



Does java parse large text file

Software Development


Reply
 
Thread Tools Search this Thread
  #1  
Old 30-07-2009
Member
 
Join Date: Nov 2008
Posts: 94
Does java parse large text file
  

I have fairly large text file in Java and the size for that file would be nearly about 20 mb. Could i be able to use the java for doing this or should i go for shell scripting? and what could be the maximum size of a file that could be parsed with java. Like for example can i parse a txt file whose size is > 20 m.b. and parsing such big file will have any impact on the perfomance.

Reply With Quote
  #2  
Old 30-07-2009
Member
 
Join Date: Mar 2008
Posts: 258
Re: Does java parse large text file

Java does not have a built in method to count the number of times a String appears. Fortunately, it's quite simple to implement one. garbage collection is something to keep in mind. If you produce hundreds of thousands of tokens from the file you're parsing, then you will need to tune the VM for max performance.
Reply With Quote
  #3  
Old 30-07-2009
Member
 
Join Date: Apr 2008
Posts: 117
Re: Does java parse large text file

The parsing is some text files that are stored in a folder and the size of the each file could be between 20 to 30 m.b ColdFusion parses XML document and it needs to able to load and parse the entire document in memory before it can return a result to you. I wouldn't dare say that without some figures to back it up. Both Scanner and String.split use a java.util.regex.Pattern object.
Reply With Quote
  #4  
Old 30-07-2009
Member
 
Join Date: Nov 2008
Posts: 94
Re: Does java parse large text file

I am using the substring method of the String class to parse out the individual fields. I read the input record into a String variable and substring out the individual fields. Some of the files have as many as 150-200 fields on a record. I have also notice that parsing is done for small file, but while passing huge size file , the JVM memory issue occurs
Reply With Quote
Reply

  TechArena Community > Software > Software Development
Tags: , , ,



Thread Tools Search this Thread
Search this Thread:

Advanced Search


Similar Threads for: "Does java parse large text file"
Thread Thread Starter Forum Replies Last Post
How to parse comma separated text into Multiple Fields? Leonard Windows Software 5 12-03-2010 05:39 AM
How to add parse in scanner in java? MAGAR Software Development 5 02-03-2010 03:30 PM
Powershell script to parse system logs in text file SADIQ Operating Systems 2 02-06-2009 07:05 PM
Batch Script Text file parse tator.usenet@gmail.com Windows Server Help 5 25-03-2009 02:12 AM
Batch Script to parse lines in text file jntoner1@gmail.com Windows Server Help 8 28-01-2009 03:38 AM


All times are GMT +5.5. The time now is 11:48 AM.