Language independent optimization of text readability formulas with deep reinforcement learning
Readability formulas are used to assess the level of difficulty of a text. These language dependent formulas are
introduced with pre-defined parameters. Deep reinforcement learning models can be used for parameter optimization. In this article
we argue that an Actor-Critic based model can be used to optimize the parameters in the readability formulas. Furthermore, a
selection model is proposed for selecting the most suitable formula to assess the readability of the input text. English and
Persian data sets are used for both training and testing. The experimental results of the parameter optimization model show that,
on average, the F-score of the model for English increases from 24.7% in the baseline to 38.8%, and for Persian from 23.5% to
47.7%. The proposed algorithm selection model further improves the parameter optimization model to 65.5% based on F-score for both
English and Persian.
Article outline
- 1.Introduction
- 2.Related Works
- 3.Proposed method
- 3.1Reinforcement learning
- 3.1.1State space
- 3.1.2Action space
- 3.1.3Rewarding
- 3.2Parameter optimization
- 3.3Algorithm selection
- 3.4Model training
- 4.Experimental results
- 4.1Dataset
- 4.1.1English dataset
- 4.1.2Persian dataset
- 4.2Setup of experiments
- 4.3Results and discussion
- 5.Conclusion and future work
- Notes
-
References
References (70)
References
Al Qundus, J., Paschke, A., Gupta, S., Alzouby, A. M., & Yousef, M. (2020). Exploring
the impact of short-text complexity and structure on its quality in social media. Journal of
Enterprise Information Management. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Amstad, T. (1978). Wie verständlich sind unsere zeitungen? [How Readable Are Our
Newspapers?] Zurich, Switzerland: University of Zurich.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Antunes, H., & Lopes, C. T. (2019). Analyzing
the adequacy of readability indicators to a non-English language. International Conference
of the Cross-Language Evaluation Forum for European
Languages, (pp. 149–155). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
Azpiazu, I. M., & Pera, M. S. (2019). Multiattentive
recurrent neural network architecture for multilingual readability assessment. Transactions of
the Association for Computational
Linguistics, 71, 421–436. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Balyan, R., McCarthy, K. S., & McNamara, D. S. (2018). Comparing
machine learning classification approaches for predicting expository text difficulty. Grantee
Submission.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bijankhan, M. (2004). The
role of corpora in writing a grammar: Introducing a software. Journal of
Linguistics, 19(2), 48–67.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bohnet, B. (2009). Efficient
parsing of syntactic and semantic dependency structures. Proceedings of the 13th Conference on
Computational Natural Language Learning: Shared
Task, (pp. 67–72). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cha, M., Gwon, Y., & Kung, H. (2017). Language
modeling by clustering with word embeddings for text readability assessment. Proceedings of the
2017 ACM on Conference on Information and Knowledge
Management, (pp. 2003–2006). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cuayáhuitl, H., Lee, D., Ryu, S., Cho, Y., Choi, S., Indurthi, S., Yu, S., Choi, H., Hwang, I., & Kim, J. (2019). Ensemble-based
deep reinforcement learning for
chatbots. Neurocomputing, 3661, 118–130. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dale, E., & Chall, J. S. (1948). A
formula for predicting readability: Instructions. Educational research
bulletin, 37–54.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dayani, M. (2000). A
criteria for assessing the Persian texts’ readability. Journal of Social Science and
Humanities, 101, 35–48.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
DuBay, W. H. (2004). The
principles of readability. Impact Information.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dueppen, A. J., Bellon-Harn, M. L., Radhakrishnan, N., & Manchaiah, V. (2019). Quality
and readability of English-language internet information for voice disorders. Journal of
Voice, 33(3), 290–296. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Eslami, M., SharifiAtashgah, M., Lamjiri, S. A., & Zandi, T. (2004). Persian
productive lexicon. Proceedings of the 1st Workshop on the Persian Language and
Computer.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Flesch, R. (1979). How
to Write Plain English: A Book for Lawyers and Consumers. Harper &
Row.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Flesch, R. (1948). A
new readability yardstick. Journal of Applied
Psychology, 32(3), 221. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Franccois, T., & Fairon, C. (2012). An
“AI readability” formula for French as a foreign language. Proceedings of the 2012 Joint
Conference on Empirical Methods in Natural Language Processing and Computational Natural Language
Learning, (pp. 466–477).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ghaffari, M., MahmoodiBakhtiyari, B., & Zolfaghari, H. (2004). Let’s
learn Persian (Vol. 1–3). Madreseh Publication.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ghayoomi, M. (2012). Bootstrapping
the development of an HPSG-based treebank for Persian. Linguistic Issues in Language
Technology, 7(1). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ghayoomi, M. (2013). Introducing
a treebank and a statistical parser for Persian. Proceedings of the 8th Conference of Iranian
Linguistics, 21, 666–679.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ghayoomi, M. (2019). Transition
from rule-based to statistical lemmatization in Persian. Proceedings of the 5th National
Conference on Computational
Linguistics, (pp. 57–86).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ghayoomi, M. (2022). Application
of computational linguistics to predict language proficiency level of Persian learners’
textbooks. Journal of Language
Horizons, 6(1), 29–52. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ghayoomi, M., & Kuhn, J. (2014). Converting
an HPSG-based treebank into its parallel dependency-based treebank. Proceedings of the 9th
International Conference on Language Resources and
Evaluation, (pp. 802–809).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Goudjil, M., Koudil, M., Bedda, M., & Ghoggali, N. (2018). A
novel active learning method using svm for text classification. International Journal of
Automation and
Computing, 15(3), 290–298. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gunning, R. (1952). The
Technique of Clear Writing. McGraw-Hill.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hafner, R., & Riedmiller, M. (2011). Reinforcement
learning in feedback control. Machine
Learning, 84(1–2), 137–169. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hausknecht, M., & Stone, P. (2015). Deep
reinforcement learning in parameterized action space. arXiv preprint
arXiv:1511.04143. [URL]
Jiang, Z., Gu, Q., Yin, Y., & Chen, D. (2018). Enriching
word embeddings with domain knowledge for readability assessment. Proceedings of the 27th
International Conference on Computational
Linguistics, (pp. 366–378).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Karačić, J., Dondio, P., Buljan, I., Hren, D., & Marušić, A. (2019). Languages
for different health information readers: Multitrait-multimethod content analysis of cochrane systematic reviews textual
summary formats. BMC Medical Research
Methodology, 19(1), 75. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Keneshloo, Y., Ramakrishnan, N., & Reddy, C. K. (2019). Deep
transfer reinforcement learning for text summarization. Proceedings of the 2019 SIAM
International Conference on Data Mining, 675–683. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kincaid, J. P., Fishburne Jr, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation
of new readability formulas (automated readability index, Fog count and Flesch reading ease formula) for navy enlisted
personnel. Technical Report. Naval Technical Training Command Millington TN Research
Branch. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kingma, D. P., & Ba, J. (2014). Adam:
A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [URL]
Klein, D., & Manning, C. D. (2003). Accurate
unlexicalized parsing. Proceedings of the 41st Meeting of the Association for Computational
Linguistics, (pp. 423–430). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous
control with deep reinforcement learning. arXiv preprint
arXiv:1509.02971. [URL]
Lively, B. A., & Pressey, S. L. (1923). A
method for measuring the “vocabulary Burden” of textbooks. Educational Administration and
Supervision, 91, 389–398.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Manek, A. S., Shenoy, P. D., Mohan, M. C., & Venugopal, K. (2017). Aspect
term extraction for sentiment analysis in large movie reviews using Gini index feature selection method and SVM
classifier. World wide
web, 20(2), 135–154. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed
representations of words and phrases and their
compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, & K. Q. Weinberger (Eds.), Advances
in neural information processing
systems 261 (pp. 3111–3119). Curran Associates, Inc. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing
Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. [URL]
Mohammadi, H., & Khasteh, S. H. (2020). A
machine learning approach to Persian text readability assessment using a crowdsourced
dataset. 2020 28th Iranian Conference on Electrical Engineering
(ICEE), 1–7. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mohammadi, H., & Khasteh, S. H. (2019). Text
as environment: A deep reinforcement learning text readability assessment model. arXiv
preprint arXiv:1912.05957. [URL]
Müller, T., Cotterell, R., Fraser, A., & Schütze, H. (2015). Joint
lemmatization and morphological tagging with lemming. Proceedings of the 2015 Conference on
Empirical Methods in Natural Language
Processing, (pp. 2268–2274). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Müller, T., Schmid, H., & Schütze, H. (2013). Efficient
higher-order CRFs for morphological tagging. Proceedings of the 2013 Conference on Empirical
Methods in Natural Language
Processing, (pp. 322–332).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Narayan, S., Cohen, S. B., & Lapata, M. (2018). Ranking
sentences for extractive summarization with reinforcement learning. arXiv preprint
arXiv:1802.08636. [URL]. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
Ngo-Ye, T. L., Sinha, A. P., & Sen, A. (2017). Predicting
the helpfulness of online reviews using a scripts-enriched text regression model. Expert
Systems with
Applications, 711, 98–110. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nuruzzaman, M., & Hussain, O. K. (2018). A
survey on chatbot implementation in customer service industry through deep neural
networks. 2018 IEEE 15th International Conference on e-Business Engineering
(ICEBE), (pp. 54–61). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pancer, E., Chandler, V., Poole, M., & Noseworthy, T. J. (2019). How
readability shapes social media engagement. Journal of Consumer
Psychology, 29(2), 262–270. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Poornamdariyan, T. (1994). The
Persian Lesson for Foreign Persian Learners (For Beginners). Institute for
Humanities; Cultural Studies Publications.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rottensteiner, S. (2010). Structure,
function and readability of new textbooks in relation to comprehension. Procedia-Social and
Behavioral Sciences, 21, 3892–3898. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
SaffarMoghaddam, A. (2003). General
Persian: Basic constructions. Council of Extending Persian Language; Linguistics at the
Institute for Humanities; Cultural Studies.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
SaffarMoghaddam, A. (2008). The
Persian language (Vol. 1–4). Council of Extending Persian Language; Linguistics at the
Institute for Humanities; Cultural Studies.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Salton, G. M., Wong, A., & Yang, C.-S. (1975). A
vector space model for automatic indexing. Communications of the
ACM, 18(11), 613–620. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Samareh, Y. (1989). Teaching
the Persian Language (Vol. 1). Alhoda International Publications.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Samareh, Y. (2005). Teaching
the Persian Language (Vol. 2–4). Alhoda International Publications.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Senter, R., & Smith, E. A. (1967). Automated
readability index (tech. rep.). CINCINNATI UNIV OH.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Serban, I. V., Sankar, C., Germain, M., Zhang, S., Lin, Z., Subramanian, S., Kim, T., Pieper, M., Chandar, S., Ke, N. R., et al. (2017). A
deep reinforcement learning chatbot. arXiv preprint arXiv:1709.02349. [URL]
Shen, C., Gonzalez, Y., Chen, L., Jiang, S. B., & Jia, X. (2018). Intelligent
parameter tuning in optimization-based iterative ct reconstruction via deep reinforcement
learning. IEEE transactions on medical
imaging, 37(6), 1430–1439. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sherman, L. (1893). Analytics
of Literature: A Manual for the Objective Study of English Prose and
Poetry. Ginn. [URL]
Silveira, N., Dozat, T., de Marneffe, M. C., Bowman, S., Connor, M., Bauer, J., & Manning, C. D. (2014). A
gold standard dependency corpus for English. Proceedings of the 9th International Conference on
Language Resources and
Evaluation, (pp. 2897–2904).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Song, S., Huang, H., & Ruan, T. (2019). Abstractive
text summarization using lstm-cnn based deep learning. Multimedia Tools and
Applications, 78(1), 857–875. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sutton, R. S., & Barto, A. G. (1998). Reinforcement
Learning: An Introduction (Vol. 135). MIT press Cambridge. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Temnikova, I., Vieweg, S., & Castillo, C. (2015). The
case for readability of crisis communications in social media. Proceedings of the 24th
International Conference on World Wide
Web, (pp. 1245–1250). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Vajjala, S., & Lučić, I. (2018). OnestopEnglish
corpus: A new corpus for automatic readability assessment and text simplification. Proceedings
of the 13th Workshop on Innovative Use of NLP for Building Educational
Applications, (pp. 297–304). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wang, Y., & Jin, H. (2019). A
deep reinforcement learning based multi-step coarse to fine question answering (MSCQA)
system. Proceedings of the AAAI Conference on Artificial
Intelligence, 331, 7224–7232. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wasike, B. (2018). Preaching
to the choir? An analysis of newspaper readability vis-a-vis public
literacy. Journalism, 19(11), 1570–1587. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine
Learning, 8(3–4), 279–292. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Watkins, C. J. C. H. (1989). Learning
from Delayed Rewards. Doctoral Dissertation. King’s College. Cambridge, UK.
Xia, M., Kochmar, E., & Briscoe, T. (2019). Text
readability assessment for second language learners. arXiv preprint
arXiv:1906.07580. [URL]
Zalmout, N., Saddiki, H., & Habash, N. (2016). Analysis
of foreign language teaching methods: An automatic readability approach. Proceedings of the 3rd
workshop on natural language processing techniques for educational applications
(NLPTEA2016), (pp. 122–130).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zarghamiyan, M. (1998). Series
of Teaching the Persian Language (From Beginner to Advanced) (Vol. 1). Council of Extending
Persian Language; Linguistics.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zarghamiyan, M. (2001). Series
of Teaching the Persian Language (From Beginner to Advanced) (Vol. 2–3). Council of Extending
Persian Language; Linguistics.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (2)
Cited by two other publications
Hadizadeh Moghaddam, Arya & Saeedeh Momtazi
2024.
A semantic modular framework for events topic modeling in social media.
Multimedia Tools and Applications 83:4
► pp. 10755 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Mayahi, Ahmad Jaber & Emman Naser Alshatti
2023.
2023 Computer Applications & Technological Solutions (CATS),
► pp. 1 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.