I get the ljuniq cookie by opening a new private browsing window, opening the network request section of the browser window, going to https://www.dreamwidth.org/ then solving the CAPTCHA. It redirects back to the homepage which returns a set-cookie HTTP header shown in the developer tools, the line looks something like
(I replaced parts of it with "x" in that since it's just an example.)
The part that looks like ewp0jLxxxx97IQp%3A171xxxx687 is the important part, I copy that and paste it as the <ljuniq> value in the configuration file. Note the %3A in the cookie I think needs to be changed to a : (since it's "URL encoded")
So the end configuration option looks like
<ljuniq>ewp0jLxxxx97IQp:171xxxx687</ljuniq>
That cookie tends to expire really easily in my experience (or maybe it's my user error, I'm bad at doing things right). I'm not sure what the conditions are, but it sometimes takes me a few tries to get it to let me download the images. If it fails, it'll say "Content type text not expected, image skipped" for the image that didn't work. I've used commenting out the logic at the end of the "Respect the global image cache setting" bit in ljdumptohtml.py and then running only ljdumptohtml.py retry the images that failed without waiting 24h or deleting the cache rows from the database manually.
Unrelated if it helps, found another crash but I'm not sure how to fix it:
The lines
print('Adding new event %s at %s: %s' % (data['itemid'], data['eventtime'], data['subject']))
and
print('Updating event %s at %s: %s' % (data['itemid'], data['eventtime'], data['subject']))
can crash with errors like UnicodeEncodeError: 'cp932' codec can't encode character '\xe2' in position 54: illegal multibyte sequence
I don't know how to fix that, but removing the printing of data['subject'] works around it.
Possible note that since it uses the original image rather than also downloading the thumbnail, the image shown in the rendered HTML page can be very large and run off the web page if the original picture is high resolution but had a small thumbnail in the journal entry. Since that is different than the patch, I'm guessing that may be a deliberate decision though.
If I read it right I think this line sends the cookie unconditionally even for non-Dreamwidth images, just to note that I don't think that follows the cookie security that a browser would have, where the cookie is private information shared only to the original host. It may be a security consideration to leak it to other hosts, though I don't know how DW uses that cookie.
Thank you again for the script, and sorry to mention a lot of things!
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Re: <3
Date: 2024-05-21 10:56 pm (UTC)I get the ljuniq cookie by opening a new private browsing window, opening the network request section of the browser window, going to https://www.dreamwidth.org/ then solving the CAPTCHA. It redirects back to the homepage which returns a set-cookie HTTP header shown in the developer tools, the line looks something like
Long comment, click to open...
(I replaced parts of it with "x" in that since it's just an example.)
The part that looks like
ewp0jLxxxx97IQp%3A171xxxx687is the important part, I copy that and paste it as the <ljuniq> value in the configuration file. Note the%3Ain the cookie I think needs to be changed to a:(since it's "URL encoded")So the end configuration option looks like
<ljuniq>ewp0jLxxxx97IQp:171xxxx687</ljuniq>That cookie tends to expire really easily in my experience (or maybe it's my user error, I'm bad at doing things right). I'm not sure what the conditions are, but it sometimes takes me a few tries to get it to let me download the images. If it fails, it'll say "Content type text not expected, image skipped" for the image that didn't work. I've used commenting out the logic at the end of the "Respect the global image cache setting" bit in ljdumptohtml.py and then running only ljdumptohtml.py retry the images that failed without waiting 24h or deleting the cache rows from the database manually.
Unrelated if it helps, found another crash but I'm not sure how to fix it:
The lines
print('Adding new event %s at %s: %s' % (data['itemid'], data['eventtime'], data['subject']))and
print('Updating event %s at %s: %s' % (data['itemid'], data['eventtime'], data['subject']))can crash with errors like
UnicodeEncodeError: 'cp932' codec can't encode character '\xe2' in position 54: illegal multibyte sequenceI don't know how to fix that, but removing the printing of data['subject'] works around it.
Possible note that since it uses the original image rather than also downloading the thumbnail, the image shown in the rendered HTML page can be very large and run off the web page if the original picture is high resolution but had a small thumbnail in the journal entry. Since that is different than the patch, I'm guessing that may be a deliberate decision though.
If I read it right I think this line sends the cookie unconditionally even for non-Dreamwidth images, just to note that I don't think that follows the cookie security that a browser would have, where the cookie is private information shared only to the original host. It may be a security consideration to leak it to other hosts, though I don't know how DW uses that cookie.
Thank you again for the script, and sorry to mention a lot of things!