Byte array to file in java without overwriting,

The code below gets a byte array from an HTTP request and saves it in bytes[], the final data will be saved in message[].

I check to see if it contains a header by converting it to a String[], if I do, I read some information from the header then cut it off by saving the bytes after the header to message[].

I then try to output message[] to file using FileOutputStream and it works slightly, but only saves 10KB of information,one iteration of the while loop, (seems to be overwriting), and if I set the FileOutputStream(file, true) to append the information, it works... once, then the file is just added on to the next time I run it, which isn't what I want. How do I write to the same file with multiple chunks of bytes through each iteration, but still overwrite the file in completeness if I run the program again?

byte bytes[] = new byte[(10*1024)];
            while (dis.read(bytes) > 0)
            {
                //Set all the bytes to the message
                byte message[] = bytes;
                String string = new String(bytes, "UTF-8");

                //Does bytes contain header?
                if (string.contains("\r\n\r\n")){
                    String theByteString[] = string.split("\r\n\r\n");
                    String theHeader = theByteString[0];
                    String[] lmTemp = theHeader.split("Last-Modified: ");
                    String[] lm = lmTemp[1].split("\r\n");
                    String lastModified = lm[0];
                    //Cut off the header and save the rest of the data after it
                    message = theByteString[1].getBytes("UTF-8");

                    //cache
                    hm.put(url, lastModified);
                }

                //Output message[] to file.
                File f = new File(hostName + path);
                f.getParentFile().mkdirs(); 
                f.createNewFile();
                try (FileOutputStream fos = new FileOutputStream(f)) {
                    fos.write(message);
                } catch (IOException ioe) {
                    ioe.printStackTrace();
                }

            }
        }
Jon Skeet
people
quotationmark

You're opening a new FileOutputStream on each iteration of the loop. Don't do that. Open it outside the loop, then loop and write as you are doing, then close at the end of the loop. (If you use a try-with-resources statement with your while loop inside it, that'll be fine.)

That's only part of the problem though - you're also doing everything else on each iteration of the loop, including checking for headers. That's going to be a real problem if the byte array you read contains part of the set of headers, or indeed part of the header separator.

Additionally, as noted by EJP, you're ignoring the return value of read apart from using it to tell whether or not you're done. You should always use the return value of read to know how much of the byte array is actually usable data.

Fundamentally, you either need to read the whole response into a byte array to start with - which is easy to do, but potentially inefficient in memory - or accept the fact that you're dealing with a stream, and write more complex code to detect the end of the headers.

Better though, IMO, would be to use an HTTP library which already understands all this header processing, so that you don't need to do it yourself. Unless you're writing a low-level HTTP library yourself, you shouldn't be dealing with low-level HTTP details, you should rely on a good library.

people

See more on this question at Stackoverflow