Saving the messages of Duolingo forums

[eI000yo]

Saving the messages of Duolingo forums

---

Maybe in a week we won't be able to see those forums anymore. Only the Sentence Discussions will remain.

There are several solutions to save them:

• Copy them to a web host.
• Copy and post them here.
• Save them in your device.

The best option to copy and save them.

---

I posted this sometimes. The best option is to save them as a JSON file.
A JSON is a specially formatted text file.. You can reuse it, e.g. as part of a javascript.

You can save them as a web page or print them as pdf, but if the post have a lot of formatting like tables, color text and so on you will have to do a lot of editing to get back the original look.
In the JSON is the original text as written by the original author.

How to go to the JSON

---

It is very easy. See the next image

Goto to a discussion, delete the word forum and the dot. Add a lowercase s after the word comment .
Remark: If there is a text after the number of the post, the title of the debate, you must remove it, even the slash.

Save the page as .json.
A short discussion could be only 50 KiB, a large one about 1 MiB.
So you can save thousands of posts in a GigaByte .

Viewing the JSON

---

The best viewer to view these files is Firefox, the web browser, even you can view and edit them using a text editor.
But you can see a lot of messy text and the accent vowel could bee displayed as a code, for example.

• Open the JSON in Firefox.
• Switch to JSON if not selected.
• Tap Collapse all or something like this.
• Expand by tapping on ▶ the message field.

That is the original message. You just have to fix something, maybe, as we see later.

In the rest of the fields we can see more details: username is original author, for example.

Seeing the other messages

---

They are in the comments field.

The best option is to tap on Expand all (slowly) or similar.
And it can really take a few minutes if you don't have a powerful device and you are opening a large document. Patience please.

Type message in Filter JSON or similar.

It will show all messages, but twice. You can also see the HTML message. This is not helpful.
The second one is the MarkDown format.

You will also see the deleted messages.
The last message is the original post. The rest of the messages are sorted by votes, I think.

Pasting messages here

---

Simply copy the desired message , without the start and end quotation marks and paste it into the editing box here in duome.

You have to replace each \n with an ENTER
If you have a large document, you can use a text editor and replace each \n with the code [br].

All other codes work here, including tables, bold text, colors, titles, etc.
Try a preview, maybe you have to fix something, usually little errors.

Making an automatic process

---

I am sure that it is possible. But it is best to post only the most interesting parts here.
It is conterproductive to copy entire discussions here.
It is possible to extract fields using a software, but is is not worth it.
A simple way could be: apply a filter, select everything and paste it to a table column, apply another filter and do the same, etc.
This way you can create a table with the message, the date and time, and the author, for example.

Your own messages

---

You can export your Duolingo Data going to https://drive-thru.duolingo.com/
You have a button that says ACCESS PERSONAL DATA

Important: On the same page, in red, it is also the button to erase your account.
It is not very dangerous because you will receive an email for confirm that. And you can undo the erasure for a few days.

Even Duolingo says that will receive your data up to 30 days, I always received it in a few hours.
You have to go to a link and download a zip file. There is a file named forum-posts.csv in the zip. It has our posts. Maybe some accented characters are written using a code. Replace them using a text editor if they are several hundreds, for example.
And the line breaks are also codes. Fix them as stated above.

What may remain of the forums

---

It is possible that some post could be found as cached in the web searchers. On Yandex I found recently a post of mine that I had deleted probably one year ago.

---

It is important that you save each post that you consider interesting.
IMO, you can post it later.

Regards!

[Basler Biker]

A very complete description and how to.

Personally I save on a local disk to start with. I am independent of archiving servers overload and wait times.
Later when I have time, I open the local page and copy paste The OP, the author, and only the most on-topic replies to OP.
These I simply paste into the new duome topic; a,ways with reference to the OP alias, and the repliers aliases.
And I keep full original texts of these.

Now I wonder, can I make the same JSON file from my local copy, as from the online DL post? Or is something lost in the process of saving locally as html? because, locally you have the local html file, and a subfolder with the same name holding all resources for that html file.

[eI000yo]

I have done a test right now:

• html 3,6 MiB
• JSON 546,3 KiB

You lose some important information if you save it as html, and you save images that are useless.
On Duolingo forum images are only links to other pages.

Of course, I save the JSON in my device. In my test html is almost 7 times heavier than json.
The message field is exactly the text typed in the box. Only ENTER^S are not.

JSON are in Duolingo servers, it is a different URL.
A parser, see https://en.wikipedia.org/wiki/Parser_(p ... _language) converts MarkDown format to HTML. Both formats are in the JSON. The saved web page is only html.

[Basler Biker]

OK. I did some test again. I have to scroll down in DL till the end to have all replies, till the scroll down stopped.
On the very first Swedish welcome page I had 8,1 MB when saved locally as html. that's indeed too much.

But now, does the JSON hold all these 504 replies without having to scroll down?

https://forum.duolingo.com/comment/5522374

[eI000yo]

You don't need to scroll, it is a text file. Just go to https://www.duolingo.com/comments/5522374

Exactly in my HD: 556,2 KiB (569.530)

You can save and extract the comment fields, if you want. I use jq

I open a shell window in the folder where is the saved JSON and I run

Code: Select all

jq -M -r '.comments[] |.message' 5522374.json 

I get a text that I printed to pdf

5522374.pdf

(74.01 KiB) Downloaded 32 times

If I run

Code: Select all

jq -M -r '.message' 5522374.json

I get the OP message:

ÄNTLIGEN!!!

We are extremely proud to present to you the fourth incubated language course from English; Swedish! It has been a long journey with a lot of ups and downs, but finally the day has arrived to launch the Beta version of the course. As it is a Beta version, the course might still be a bit rough around the edges, but please be patient and continue reporting any problems that you encounter so that we can polish and improve the course.

The Swedish forum, which will be monitored by Anrui, Arnauti, Anders91 & Blehg, can be found among the other forums and you can post in this new forum by going to Swedish Discussions and clicking the green button New discussion. Make sure you follow Duolingo’s guidelines at all times! Posts that are not in line with the guidelines will be removed.

We have provided tips & notes for several skills. However, since the course has just been launched, tips & notes for some skills are still missing. Please, bear with us as we will add these over the coming weeks. If you run into problems and the tips & notes don’t provide the answers you need, there are always helpful members just a few clicks away in the forums and threads.

We, the Swedish team, are very well aware of the problems with the TTS (Text-To-Speech) system which has been implemented so far. We are trying desperately to find a better solution, but unfortunately, it seems to be quite difficult to integrate other TTS systems. Until we have figured out how to employ another TTS, we encourage you to report all sentences that seem to be badly pronounced. We have been forced to disable the voice function in many cases since we do not want to teach incorrect pronunciation. However, a few common words have been kept even though their pronunciation is not entirely correct.

The first one is “De” which means “They”. Even though the words is spelled “de”, it is pronounced “dom” in all cases. The objective pronoun “dem” (them) is also pronounced “dom”, but at least the TTS gets this right!

Another example is “är” which means “is”. The TTS pronounces this in the way people in the region around Stockholm would, i.e. “é”. There is a lot of variation in the country when it comes to the pronunciation of this word, but the pronunciation that is closest to a standard is “är” (as it is spelled), therefore we would suggest that you stick to that even though the TTS says otherwise.

Lycka till!
/Anders91, Anrui, Arnauti & Blehg

[Basler Biker]

Very nice ! Thanks.

[Corinnebelle]

Note: When doing this for sentence discussions you need to also remove the slash / at the end and the sentence as well.

I.e.
https://forum.duolingo.com/comment/1652 ... גבניה-שלךָ

becomes

https://www.duolingo.com/comments/16526362

[Deleted User 1400]

Note: When doing this for sentence discussions you need to also remove the slash / at the end and the sentence as well.

I.e.
https://forum.duolingo.com/comment/1652 ... גבניה-שלךָ

becomes

https://www.duolingo.com/comments/16526362

Just shortening the URL? Yeah that's an easy one liner..

[Corinnebelle]

@rtqs-sandra You also have to do the things in @eI000yo first comment. Delete forum and the dot afterwards and add an s to comment.

[Deleted User 1400]

@rtqs-sandra You also have to do the things in @eI000yo first comment. Delete forum and the dot afterwards and add an s to comment.

Ah I see, but still one line, altogether, basically just a regex.