There's a file that I would like to make sure does not grow larger than 2 GB (as it must run on a system that uses ext 2). What's a good way to check a file's size bearing in mind that I will be writing to this file in between checks? In particular, do I need to worry about buffered, unflushed changes that haven't been written to disk yet?
9
-
2Is there a reason you can't just keep track of the file size yourself - that is, see what the size is when you open it and increment a counter when you write? Not particularly elegant, but it should work.Blair Conrad– Blair Conrad2009-12-08 14:39:23 +00:00Commented Dec 8, 2009 at 14:39
-
I suppose that's a possibility I hadn't thought of... I might try that as well.Jason Baker– Jason Baker2009-12-08 14:40:49 +00:00Commented Dec 8, 2009 at 14:40
-
Is that not inefficient as hell though?Dominic Bou-Samra– Dominic Bou-Samra2009-12-08 14:51:44 +00:00Commented Dec 8, 2009 at 14:51
-
2The maximum file size limit under ext2 is 16GiB -- 64TiB depending on the block size. See en.wikipedia.org/wiki/Ext2. This doesn't answer your question, but just thought this might be helpful.unutbu– unutbu2009-12-08 14:53:47 +00:00Commented Dec 8, 2009 at 14:53
-
1Jason, what would happen if you let the file grow too large? Generally in Python, try not to "look before you leap"... let exceptions occur, and handle them then. Usually faster and cleaner. What would you do if your counter said the file was about to become too large? Can you do the same after catching an exception when it does get too large? Some extra detail might help in your question.Peter Hansen– Peter Hansen2009-12-08 14:58:14 +00:00Commented Dec 8, 2009 at 14:58
|
Show 4 more comments
7 Answers
Perhaps not what you want, but I'll suggest it anyway.
import os
a = os.path.getsize("C:/TestFolder/Input/1.avi")
Alternatively for an opened file you can use the fstat function, which can be used on an opened file. It takes an integer file handle, not a file object, so you have to use the fileno method on the file object:
a = open("C:/TestFolder/Input/1.avi")
b = os.fstat(a.fileno()).st_size
answered Dec 8, 2009 at 14:38
Dominic Bou-Samra
15.5k28 gold badges108 silver badges165 bronze badges
Comments
os.fstat(file_obj.fileno()).st_size should do the trick. I think that it will return the bytes written. You can always do a flush before hand if you are concerned about buffering.
1 Comment
personal_cloud
And works in append mode too! Thank you. And yeah, I would flush before calling this.
Though this is an old question, I think that Isak has the simplest solution. Here's how to do it in Python:
# Assuming f is an open file
>>> pos = f.tell() # Save the current position
>>> f.seek(0, 2) # Seek to the end of the file
>>> length = f.tell() # The current position is the length
>>> f.seek(pos) # Return to the saved position
>>> print length
1024
4 Comments
Jkm
I think that in the first line (save current position), you should use f.tell(), not the seek(), which would cause an exception since seek() needs at least 1 argument.
Trenton
@Jkm Yes, you are right! Not sure how I missed that. Thanks!
personal_cloud
This will calculate the file size correctly, but won't restore the position correctly due to known issues with tell in append mode.
Trenton
@personal_cloud I thought that would not be an issue as long as you did not write between the
tell and seek, but I may be wrong. I didn't have an issue in my tests, but it looks like those issues vary by platform. Thanks for pointing that out.You could start with something like this:
class TrackedFile(file):
def __init__(self, filename, mode):
self.size = 0
super(TrackedFile, self).__init__(filename, mode)
def write(self, s):
self.size += len(s)
super(TrackedFile, self).write(s)
Then you could use it like this:
>>> f = TrackedFile('palindrome.txt', 'w')
>>> f.size
0
>>> f.write('A man a plan a canal ')
>>> f.size
21
>>> f.write('Panama')
27
Obviously, this implementation doesn't work if you aren't writing the file from scratch, but you could adapt your __init__ method to handle initial data. You might also need to override some other methods: writelines, for instance.
This works regardless of encoding, as strings are just sequences of bytes.
>>> f2 = TrackedFile('palindrome-latin1.txt', 'w')
>>> f2.write(u'A man a plan a canál '.encode('latin1')
>>> f3 = TrackedFile('palindrome-utf8.txt', 'w')
>>> f3.write(u'A man a plan a canál '.encode('utf-8'))
>>> f2.size
21
>>> f3.size
22
3 Comments
Bite code
That's not actually. It you use ASCII, ISO1559 and UTF-8, the result will be the same, but the on disk size will not be.
jcdyer
No. It works for other encodings too, if you use actual strings. Answer modified to demonstrate.
jcdyer
The trick is you can't just write unicode objects and rely on the os's encoding.
I'm not familiar with python, but doesn't the stream object (or whatever you get when opening a file) have a property that contains the current position of the stream?
Similar to what you get with the ftell() C function, or Stream.Position in .NET.
Obviously, this only works if you are positioned at the end of the stream, which you are if you are currently writing to it.
The benefit of this approach is that you don't have to close the file or worry about unflushed data.
2 Comments
hurfdurf
'filehandle.tell()' indeed shows the number of bytes in the opened file, and works in either write or append mode. Not sure why all these more complex answers got upvoted.
personal_cloud
@hurfdurf No,
f.tell() does not seem to work reliably in append mode. Unless you first f.seek(0,2). I have no idea why.Or, if the file is already open:
>>> fsock = open('/etc/hosts', 'rb').read()
>>> len(fsock)
444
That's how many bytes the file is.
Comments
Most reliable would be create a wrapping class which would check file's size when you open it, track write and seek operations, count current size based on those operations and prevent from exceeding size limit.
Comments
