I have a text file with about 3 million lines and I need to replace the first two characters of every line with "". My current code is
String[] lines = content.split("\n");
content = "";
for(int i = 0;i<lines.length;i++){
System.out.println(i);
lines[i].substring(2);
content = content + lines[i];
}
It takes it about 3-4 days to finish. Is there a faster way of doing this?
My entire code:
try {
String content = readFile("text.txt", StandardCharsets.ISO_8859_1);
content = content.replaceAll("\\d","");
content = content.replaceAll("\\.","");
String[] lines = content.split("\n");
content = "";
for(int i = 0;i<lines.length;i++){
System.out.println(i);
lines[i].substring(2);
content = content + lines[i];
}
PrintWriter out = new PrintWriter("texty text.txt");
out.println(content);
} catch (IOException e) {
e.printStackTrace();
}
Is there a faster way of doing this?
Yes - don't perform repeated string concatenation. That's the part that's killing your performance. Use a StringBuilder
instead.
It's also not even doing what you want it to - you're not using the result of calling substring
.
I suspect you want something like this:
String[] lines = content.split("\n");
StringBuilder builder = new StringBuilder();
for (String line : lines) {
builder.append(line.substring(2));
// I suspect you want this, otherwise you're losing line breaks.
builder.append("\n");
}
String result = builder.toString();
Additionally, you should consider reading the file line by line instead of reading the whole thing and then splitting it. For example:
StringBuilder builder = new StringBuilder();
try (BufferedReader reader = ...) {
String line;
while ((line = reader.readLine()) != null) {
builder.append(line.substring(2));
builder.append("\n");
}
}
String result = builder.toString();
And unless you really need the whole thing in memory at any one time at all, write the lines as you process them:
try (BufferedReader reader = ...,
BufferedWriter writer = ...) {
String line;
while ((line = reader.readLine()) != null) {
// Perform any other manipulations here...
writer.write(line.substring(2));
writer.write("\n");
}
}
See more on this question at Stackoverflow