Backup of likes doesn't stop at expected count

[aretekzs]

I just installed tumblr-utils and, trying to download all my likes, it just keeps running even after passing the expected posts. Example:

blogname: Getting posts 1050 to 1099 (of 1410 expected)

blogname: Getting posts 1100 to 1149 (of 1410 expected)         
     
blogname: Getting posts 1150 to 1199 (of 1410 expected)        
      
blogname: Getting posts 1200 to 1249 (of 1410 expected)       
       
blogname: Getting posts 1250 to 1299 (of 1410 expected)        
      
blogname: Getting posts 1300 to 1349 (of 1410 expected)        
      
blogname: Getting posts 1350 to 1399 (of 1410 expected)           
   
blogname: Getting posts 1400 to 1449 (of 1410 expected)              

blogname: Getting posts 1450 to 1499 (of 1410 expected)           
   
blogname: Getting posts 1500 to 1549 (of 1410 expected)           
   
blogname: Getting posts 1550 to 1599 (of 1410 expected)              

blogname: Getting posts 1600 to 1649 (of 1410 expected)             
 
blogname: Getting posts 1650 to 1699 (of 1410 expected`)

[cebtenzzre]

If it's caused by a regression, I have a suspicion commit 4961a2f (script here) would work better for you. But chances are https://github.com/aggroskater/tumblr-utils still has better support for likes.

[cebtenzzre]

It's possible that this condition is failing:

tumblr-utils/tumblr_backup.py

Line 223 in da3370c

return doc if doc.get('meta', {}).get('status', 0) == 200 else None

You could try replacing that line with this more verbose code:

if doc.get('meta', {}).get('status', 0) != 200:
    sys.stderr.write('API response has non-200 status:\n{}\n'.format(doc))
    return None
return doc

[bbolli]

I don't get what the more verbose code does differently, ecxept for printing the document if the response code is not 200. How would this help in this situation?

[aretekzs]

If it's caused by a regression, I have a suspicion commit 4961a2f (script here) would work better for you. But chances are https://github.com/aggroskater/tumblr-utils still has better support for likes.

I tried this commit and it did indeed finish and created the html file. However, it clearly did not save all my likes so it's still not working for me.

[aretekzs]

Actually, it now seems more likely to me that this condition is failing:

tumblr-utils/tumblr_backup.py

Line 223 in da3370c

return doc if doc.get('meta', {}).get('status', 0) == 200 else None

Chances are your likes are private (not yet supported, see #128), but maybe something else is going on. (edit: this would yield an HTTP Error 401, so it wouldn't explain this). You could try replacing that line with this more verbose code:

if doc.get('meta', {}).get('status', 0) != 200:
    sys.stderr.write('API response has non-200 status:\n{}\n'.format(doc))
    return None
return doc

I actually have some code like that in my own version of the script.

This did not work either.

[cebtenzzre]

@bbolli If that None return was causing the backup loop to "try the next batch", it would be good to know what the response status was for debugging reasons (even if HTTP status was 200). But I forgot that the loop skips single posts now, so that wouldn't cause this.

[cebtenzzre]

@sldx12 If the older commit worked for you then try the latest version of tumblr-utils, which has a potential fix for this issue. If neither gets all your likes then aggroskater's might -- it walks them by timestamp instead of offset.

[aretekzs]

@cebtenzzre none of those worked. The older commit keeps having the same bug and aggroskater's one doesn't get all likes.

[cebtenzzre]

Does the latest version (download it fresh from GitHub or update your clone if you made one) still try to download more likes than expected? If that's fixed, we can close this issue and open a new one for not downloading all of the likes.

[aretekzs]

Yes, the latest version still tries to download more likes than expected.