Request export sentences list

[DragonOfTheWest]

Dear language enthousiasts developers,

When I go to the website below for practicing Chinese from duome.eu, I see that there are 1309 sentences available:
https://duome.eu/practice/en/zs

Is it possible to get a list of all of these sentences or export them to Excel for example?
This would help me a lot with using it as input into other language learning programs.

Thank you in advance for your response.

Kind regards.

[duome]

Is it possible: yes, I sent you a personal message regarding this.

[rymca]

Hello! i need and information, it's possibile to download my vocabulary words for import in third-party program programs?
Thanks you very much

[duome]

1) what language pair?
2) your personal vocabulary on day one would be zero words - are you interested in the entire vocbulary or your personal vocabulary for today (and then any other day in the future - on demand)?

[rymca]

1) what language pair?
2) your personal vocabulary on day one would be zero words - are you interested in the entire vocbulary or your personal vocabulary for today (and then any other day in the future - on demand)?

My base language is italian, but i study english in duolingo.
I would like download my progressive words, all the words exising in the Words section in APP
Thanks!

[rymca]

1) what language pair?
2) your personal vocabulary on day one would be zero words - are you interested in the entire vocbulary or your personal vocabulary for today (and then any other day in the future - on demand)?

My base language is italian, but i study english in duolingo.
I would like download my progressive words, all the words exising in the Words section in APP
Thanks!

Are there any news about it? or is it not possibile to export data required?
Thank you!

[duome]

It will be possible soon

[rymca]

It will be possible soon

It's awesome!!

[Basler Biker]

It will be possible soon

It's awesome!!

Just a reminder to all readers, by the way, that this forum is made possible by duome.
And that we, duome and the mods here, all volunteers, still have our daily jobs to do.

And at whatever speed we deliver things, even if it takes some time, it will be miles faster than DL ever did.
Thanks you for your understanding.

[antonmo]

Very excited about this as well.

Please make a how-to to import into anki

[rymca]

Is it possible: yes, I sent you a personal message regarding this.

Can you also explain to me how i can export the sentences?

[Basler Biker]

Apart from what our forum members are collecting in sentence discussion folders like this one, there used to be an api endpoint for duolingo sentences which is no longer available. I can share what I collected previously, but I don't have it for every language pair, so sometimes there would be a bit more sentences, sometimes a bit less, and sometimes there would be none. You can have an idea if you try using this pattern: https://duome.eu/practice/en/sv - change language codes, note sentence counts.

I would like it to be more than just a plain page that's only good for copy and paste though. This is why I didn't make it a standard feature.

[antonmo]

I think what is asked here is what is in the forum sentence discussions, to be available as export in a format that is easily transferable to anki

Are you working on this?

[Tomass]

Im working for duolingo scholls student test english

[PtolemysXX]

Can you also explain to me how i can export the sentences?

In case it helps you here is a user script that you can install in a browser extension like TamperMonkey or ViolentMonkey.
The script will create button "Save sentences" inside a duolingo lesson. You do not need to go through the lesson - just start it and hit the "Save" button. It will save the list of sentences for that lesson in a file that you can import into anki. This way you can collect a list of phrases from the entire course.

script

Code: Select all

// ==UserScript==
// @name			Save DL Sentences
// @namespace		http://tampermonkey.net/
// @version			0.0.1
// @description		Save Duolingo sentences for a lesson
// @author			credit goes to https://forum.duome.eu
// @match			https://www.duolingo.com/
// @match			https://www.duolingo.com/*
// @run-at			document-start
// @icon			https://www.google.com/s2/favicons?sz=64&domain=duolingo.com
// @grant			none
// ==/UserScript==

;(() => {
    const debug = false;

const console = (/** @param {Console} c */ (c) => {
	for (const k in window.console) {
		c[k] = window.console[k]
	}

	return c
})({})

let session = null;
let link = document.createElement("a");

const ankiHeader = `#separator:tab
#html:false
#deck column:1
#tags column:5
`
	/**
	 * @param {(response: AjaxResponse) => void} listener
	 *
	 * modified from https://stackoverflow.com/questions/24555370/how-can-i-catch-and-process-the-data-from-the-xhr-responses-using-casperjs/58168312#58168312
	 * and https://blog.logrocket.com/intercepting-javascript-fetch-api-requests-responses/
	 */
	const addAjaxListener = (listener) => {
		// fetch
		const originalFetch = window.fetch

	window.fetch = async (...args) => {
		const res = await originalFetch(...args)
		const { url, status, ok, headers } = res
		const clone = res.clone()
		const getBody = (type = 'json') => clone[type]()
		const getHeaders = () => Promise.resolve(headers)
		listener({ url, status, ok, headers, getBody, getHeaders })
		return res
	}

	/** @this {XMLHttpRequest} */
	const onLoadHandler = function () {
		const xhr = this
		if (xhr.readyState === 4) {
			const { responseURL: url, status } = xhr
			const getBody = () => Promise.resolve(xhr.response)
			const getHeaders = () => Promise.resolve(
				new Headers(xhr.getAllResponseHeaders()
					.trim()
					.split(/[\r\n]+/)
					.map(line => line.split(/: /))
					.map(([k, v]) => [k.trim(), v.trim()]))
			)
			listener({ url, ok: status < 400, status, getBody, getHeaders })
		}
	}

	const { open, send } = XMLHttpRequest.prototype

	XMLHttpRequest.prototype.open = function (...args) {
		this.requestUrl = args[1]
		open.apply(this, args)
	}

	XMLHttpRequest.prototype.send = function (...args) {
		const xhr = this

		if (xhr.addEventListener) {
			xhr.removeEventListener('readystatechange', onLoadHandler)
			xhr.addEventListener('readystatechange', onLoadHandler, false)
		} else {
			let handler

			const readyStateChange = (...args) => {
				if (handler) {
					if (handler.handleEvent) {
						handler.handleEvent.apply(xhr, args)
					} else {
						handler.apply(xhr, args)
					}
				}
				onLoadHandler.apply(xhr, args)
				setReadyStateChange()
			}

			const setReadyStateChange = () => {
				setTimeout(() => {
					if (xhr.onreadystatechange !== readyStateChange) {
						handler = xhr.onreadystatechange
						xhr.onreadystatechange = readyStateChange
					}
				}, 1)
			}

			setReadyStateChange()
		}
		send.apply(xhr, args)
	}
}

/* === main logic === */

const Selector = {
	NextButton: '[data-test="player-next"]',
	Blame: '[data-test*="blame"]',
	ChallengeControls: `#${CSS.escape('session/PlayerFooter')}`,
}

const sessionsUrl = 'https://www.duolingo.com/2017-06-30/sessions'
const sessionsUrlMatcher =
	new RegExp(`^${sessionsUrl.replace(/\d/g, '\\d')}\/?$`)

const observer = new MutationObserver((mutations) => {
	if (!session) return

	const {
		challenges,
		adaptiveChallenges,
		adaptiveInterleavedChallenges: a,
	} = session
})

;(async () => {
	let tries = 0
	while (++tries <= 100) {
		const root = document.body
		if (root) {
			observer.observe(root, { childList: true, subtree: true })
			break
		} else {
			await new Promise(res => setTimeout(res, 10))
		}
	}
	if (tries > 1) {
		console.warn(`tried to find root element ${tries} times`)
	}
})()

addAjaxListener(async (res) => {
	if (sessionsUrlMatcher.test(res.url)) {
		session = window.session = await res.getBody()
	}
})

const savePhrases = async () => {
    if (session == null || session.challenges == null) {
        return;
    }

    const phraseSet = [];
    phraseSet.push(ankiHeader);
    let chunks = document.URL.split("/");
    let unit_nr = chunks[chunks.length - 3];
    let unit_nr_string = "xxx";
    if (!isNaN(unit_nr)) {
        unit_nr_string = new String(unit_nr).padStart(3, '0');
    } else {
        unit_nr_string = chunks[chunks.length-2];
    }

    let anki_tag = "DL_" + session.learningLanguage + "_" + session.fromLanguage + "_" + unit_nr_string;
    let deck_name = "DL_" + session.learningLanguage + "_from_" + session.fromLanguage;

    for (let i = 0; i < session.challenges.length; i++) {
        let challenge = session.challenges[i];
        let tts = challenge.solutionTts ? challenge.solutionTts : challenge.tts;
        let chType = challenge.challengeGeneratorIdentifier.specificType;
        let t_source = challenge.metadata.text;
        let t_target = challenge.metadata.translation;
        let add_phrase = true;
        switch (chType) {
            case "dialogue":
            case "definition":
            case "select_transcription":
            case "character_select":
            case "character_match":
            case "gap_fill":
            case "match":
            case "listen_match":
            case "assist":
            case "read_comprehension":
            case "name_example":
                add_phrase = false;
                break;
                // done to here
            case "reverse_tap":
            case "tap_gap":
            case "listen_isolation":
                break;
                // done from here
            case "name":
                t_target = challenge.challengeResponseTrackingProperties.best_solution;
                t_source = challenge.metadata.hint;
                tts = "no_audio";
                break
            case "listen":
            case "listen_complete":
                t_target = challenge.metadata.text;
                t_source = challenge.metadata.solution_translation;
                break;
            case "tap":
            case "speak":
                t_target = challenge.metadata.text;
                t_source = challenge.metadata.translation;
                break;
            case "listen_tap":
                t_target = challenge.metadata.text;
                t_source = challenge.metadata.solution_translation;
                break;
            case "tap_complete":
                t_target = challenge.metadata.text;
                t_source = challenge.metadata.best_translation;
                tts = "no_audio";
                break;
            case "complete_reverse_translation":
                t_target = challenge.challengeResponseTrackingProperties.best_solution;
                tts = "no_audio";
                break;
            case "reverse_translate":
            default:
                break;
        }
        if (add_phrase) {
            //phraseSet.push([t_source, t_target, tts, chType].join("\t") + "\n");
            phraseSet.push([deck_name + "::" + unit_nr_string, t_source, t_target, /* tts, chType, */ anki_tag].join('\t') + '\n');
        }
    }

    if (phraseSet.length == 0) {
        return
    }
    const file = new Blob(phraseSet, { type: 'text/plain' });
    link.href = URL.createObjectURL(file);
    link.download = "DL_" + session.learningLanguage + "_from_" + session.fromLanguage + "_" + unit_nr_string + "_phrases.csv";
    link.click();
    URL.revokeObjectURL(link.href);
}

function renderBtnForAudio () {
    var saveSent = document.getElementById("save_sent");
    if (saveSent != null) {
        return;
    }

    const btn_next = document.querySelectorAll('[data-test="player-next"]')[0];

    if (btn_next != undefined) {
        const buttonStyle = `
        min-width: 150px;
        font-size: 17px;
        border:none;
        border-bottom: 4px solid #58a700;
        border-radius: 18px;
        padding: 13px 16px;
        transform: translateZ(0);
        transition: filter .2s;
        font-weight: 700;
        letter-spacing: .8px;
        background: #55CD2E;
        color:#fff;
        margin-left:20px;
        cursor:pointer;
        `;
        //alert("adding button");
        try {
            saveSent = document.createElement('button');

            const wrapper = btn_next.parentNode;
            wrapper.style.display = "flex";
            //Object.assign(saveSent, btn_next);

            saveSent.innerHTML = "Save Sentences";
            saveSent.id = "save_sent";
            saveSent.disabled = false;
            saveSent.style.cssText = buttonStyle;
            saveSent.className = btn_next.ClassName;
            saveSent.addEventListener("mousemove", () => {
                saveSent.style.filter = "brightness(1.1)";
            });

            saveSent.addEventListener("mouseleave", () => {
                saveSent.style.filter = "none";
            });
            wrapper.appendChild(saveSent);

            saveSent.addEventListener('click', savePhrases);
        } catch (e) {
            alert("creating button failed");
        }


    } else {
        console.log("next button not found!");
    }
}

window.originalConsole = console
window.session = session

setInterval(renderBtnForAudio, 3000);

})()

sample output

Code: Select all

#separator:tab
#html:false
#deck column:1
#tags column:5
DL_en_from_it::058	Ciao, come posso aiutarla?	Hello, how can I help you?	DL_en_it_058
DL_en_from_it::058	un'azienda internazionale	an international company	DL_en_it_058
DL_en_from_it::058	Questo è un regalo?	Is this a present?	DL_en_it_058
DL_en_from_it::058	Usa un sacchetto di plastica trasparente.	Use a clear plastic bag.	DL_en_it_058
DL_en_from_it::058	Vorrebbe aggiungere più francobolli?	Would you like to add more stamps?	DL_en_it_058
DL_en_from_it::058	Non ho bisogno di sessanta scatole!	I don't need sixty boxes!	DL_en_it_058
DL_en_from_it::058	Grandpa, do you need any envelopes?	Nonno, ti servono delle buste?	DL_en_it_058
DL_en_from_it::058	Vorrebbe questo biglietto di auguri rosa?	Would you like this pink card?	DL_en_it_058
DL_en_from_it::058	They only sell small envelopes.	Vendono solo buste piccole.	DL_en_it_058
DL_en_from_it::058	Lei vorrebbe pagare con un assegno.	She would like to pay with a check.	DL_en_it_058
DL_en_from_it::058	Are sixty cards enough?	Sessanta biglietti di auguri sono abbastanza?	DL_en_it_058
DL_en_from_it::058	Invii spesso lettere a tuo nonno?	Do you often send your grandpa letters?	DL_en_it_058

An import to Anki should work without problems, the only thing you need to do is to pick the deck name from the "Tags" pulldown - I do not know how to automate it. Here is a sample screenshot from the import dialog window:

screenshot

anki import.PNG (49.18 KiB) Viewed 4252 times

[littlegaychris]

Is it possible: yes, I sent you a personal message regarding this.

Would you mind sending me that message as well?

[duome]

Which language pair?

[littlegaychris]

Which language pair?

The same, https://duome.eu/practice/en/zs

[PtolemysXX]

You can find a comprehensive list of phrases from the Chinese course here:

viewtopic.php?t=2823-chinese-sentence-list

[littlegaychris]

You can find a comprehensive list of phrases from the Chinese course here:

viewtopic.php?t=2823-chinese-sentence-list

That list is great, thank you!

Looking forward to this being something we can export, there’s like six other language I would love this for.

[PtolemysXX]

What languages are you interested in? Have you tried to use the script from this post?

[littlegaychris]

What languages are you interested in? Have you tried to use the script from this post?

French, Spanish, Russian, Japanese, German, Portuguese, and Italian.

I haven't tried the script. I've already completed most of those courses on Duolingo, so I was hoping for a way to quickly export those sentence lists to add to Anki just as review, as the daily refresh on Duolingo itself takes so long, and they rarely make you actually produce the sentences yourself.

[PtolemysXX]

Out of those I could share a deck with selected Russian phrases (1501) and Spanish (a wild selection of ~ 2500 phrases). I am thinking of running through the Italian course to grab phrases with the script but that project is still in the "thinking" phase.

[Krabosil]

Is it possible: yes, I sent you a personal message regarding this.

Hello, I would like to request a sentence export as well if it's not too much trouble ^-^
I'm learning spanish from english

[hayek66]

Is it possible: yes, I sent you a personal message regarding this.

Hi,

Thank you! Could you please send me the message as well, if possible?

Kind regards

[hayek66]

Out of those I could share a deck with selected Russian phrases (1501) and Spanish (a wild selection of ~ 2500 phrases). I am thinking of running through the Italian course to grab phrases with the script but that project is still in the "thinking" phase.

Hi,

Thank you for sharing those decks — that’s really helpful!
I was wondering if you also have the French sentences from Duolingo, or if you plan to extract them as well?

Best,
Hayek

[PtolemysXX]

I was wondering if you also have the French sentences from Duolingo, or if you plan to extract them as well?

No, unfortunately not. I did not do the French course on Duolingo and since my last post I also deleted my Duolingo account. Sorry for that!

[Quereilla]

Is it possible: yes, I sent you a personal message regarding this.

And can I download the sentence list of Czech from English?