Draft quality warning

docs/Draft-Generation/understanding_draft_quality.md

@@ -0,0 +1,40 @@
+---
+title: Understanding Draft Quality
+sidebar_position: 1
+slug: /understanding-draft-quality
+---
+
+## Elements of Draft Quality
+
+Only human translators can decide whether a draft serves a team’s needs. However, it is helpful to understand some factors that can influence the quality of a generated draft.
+
+- Amount and quality of training data (language examples)
+  - More examples contribute to better draft quality.
+  - Examples that are well-checked and consistent improve draft quality.
+  - Examples that are relevant to the book or books to be drafted improve draft quality; for example, including examples in the same genre or with similar content is often helpful.
+- Relationship between source and target languages
+  - A closer linguistic relationship and typological match often results in better draft quality. 
+- Relationship between source text and target texts
+  - When the translation closely follows the words of the reference text, this can make model training more successful and improve draft quality. 
+  - For this reason, including a back translation in the source language as a second reference text may improve quality. 
+- Representation of source language in the language model
+  - Using a high-resource source language, that is, one which is a major modern language and has plentiful examples in the underlying language model, can improve draft quality.
+- Characteristics of target project
+  - A script unknown to the underlying language model may decrease draft quality.
+  - Agglutinative languages or those without word breaks may experience decreased draft quality.
+  - A project that translates at the paragraph level, uses many long verse ranges, or includes significant explication not found in the source text may see decreased draft quality.
+
+## Improving Draft Quality {#92f8d5c800084e4eb113511f74e26927}
+
+Here are some ideas to improve draft quality. Not all of these will be possible or relevant for all projects.
+- Use as much translated Scripture text as possible for training the model. 
+- Consider adding parallel linguistic data in the source and target languages.
+- Test different reference texts, including higher-resource or more closely-related source languages.
+- Include a back translation in the source language as a second reference text.
+- Correct any frequent spelling, keyboarding, or vocabulary inconsistencies in the translation.
+- Continue with manual translation and test draft generation again when more translated Scripture is complete.
+- See the note about [Incremental Draft Generation](https://help.scriptureforge.org/generating-a-draft#select-the-books-to-draft) and consider using this method for long books or when beginning a new genre. 
+
+## Draft Quality Warning {#9a12a839f3634e75acd511f4b426eb86}
+
+Scripture Forge uses automated scoring to estimate draft quality, and will present a warning to users if the estimated quality of a generated draft is very low. This may indicate that there is an error in the configuration that resulted in unsuccessful model training. Please see the FAQ for more information. Some drafts may be low quality even if they are not flagged by the warning. It is important to check the quality of each draft carefully.

docs/faq.md

@@ -73,3 +73,34 @@ slug: /faq
 
 	_When you make changes to your draft, Scripture Forge saves a version of those changes that you may access via the history tab. You may revert your project to an older version from here by clicking the "Restore this version" button._
 
+## Questions on draft quality warnings {#f47ac10b58cc4372a5670e02b2c3d479}
+
+1. Why do I see a warning that my draft may have lower than normal quality? What should I do if I see this warning?
+
+	_Scripture Forge uses automated metrics to determine whether model training was successful enough to prepare a draft of the requested books. Sometimes these metrics indicate that draft quality may be poor._
+
+	_The most common reason for this warning is that there is not enough example text of the kind needed to train the model. Usually, this situation results from an error in setting up the draft request. Use the following checklist as a guide to check your draft generation configuration._
+   - _Project: Are you generating the draft from a project that includes all completed books in the project?_ 
+      - _It is important not to request drafts from a project used to store AI drafts._
+      - _If some translated books, such as the New Testament, are stored in a separate Paratext project, you may need to create a new project and import all translated books to that project. You can use that project for draft generation._ 
+   - _Language codes: Does the language code for each reference and source project match the text of that project?_ 
+      - _You can update language codes in Paratext under “Project properties.”_
+   - _Draft Books: Are the book or books to draft selected in Step 2, “Select books to draft”?_ 
+   - _Training Books: Are all the completed and checked books in the project selected in Step 3, “Select books to train on”?_ 
+      - _Be sure all the books used for training have been checked by the team. They should be generally consistent in spelling, character use (keyboarding), dialect, and vocabulary._
+      - _Be sure the reference text contains all the books used for training, and that they are selected at the bottom of Step 3._
+    - _Back Translations: Are you using a back translation as a reference project? If so, check these things:_
+      - _Be sure the back translation language code represents the language of the back translation and not of the main translation project._ 
+      - _If any books in the back translation project are in a different language than the rest of the back translation, unselect those books from the back translation area at the bottom of Step 3._
+   - _Additional Data: Is it formatted correctly?_
+      - _If you uploaded additional data for model training on the “Configure sources” page, be sure that each row of that file contains sentences with the same meaning, with the source language on the left and the target language on the right._ 
+
+2. Why do I still see a low draft quality warning after completing all the checks?
+
+   _Remember that draft quality is only an estimate and may not reflect human judgment. Please review the draft carefully to determine whether it is useful in your translation work. The help text about [Understanding Draft Quality](https://help.scriptureforge.org/understanding-draft-quality) may be helpful._
+
+   _If draft quality is still low after completing the recommended checks, you may need to translate more text before generating a draft. Early in a translation project, the choice of which book to draft next is important. For example, a model trained on the book of Luke is better prepared to draft Mark than to draft Romans. If you are working in the New Testament, please review the [Guidelines for New Testament Draft Generation](https://help.scriptureforge.org/new-testament-drafting) to learn about how book sequence can affect draft quality._
+
+3. What should I do if only some of the books I generated have a low draft quality warning?
+
+   _A warning on any draft is a good reason to consider the checklist above. However, it may also be the case that your training data was a better fit for drafting some books than others. It is important to review each book draft carefully, especially those with a low quality warning, before deciding whether to use it in your translation process._

understanding_draft_quality.md

@@ -0,0 +1 @@
+