<?xml version="1.0" encoding="ISO-8859-1"?><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<front>
<journal-meta>
<journal-id>1646-9895</journal-id>
<journal-title><![CDATA[RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação]]></journal-title>
<abbrev-journal-title><![CDATA[RISTI]]></abbrev-journal-title>
<issn>1646-9895</issn>
<publisher>
<publisher-name><![CDATA[AISTI - Associação Ibérica de Sistemas e Tecnologias de Informação]]></publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id>S1646-98952023000200058</article-id>
<article-id pub-id-type="doi">10.17013/risti.50.58-72</article-id>
<title-group>
<article-title xml:lang="es"><![CDATA[Sistemas Automáticos de Calificación de Tareas de Programación: Revisión y Crítica]]></article-title>
<article-title xml:lang="en"><![CDATA[Automatic Grading Systems for Programming Assignments: Review and Critique]]></article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname><![CDATA[Cares]]></surname>
<given-names><![CDATA[Carlos]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname><![CDATA[Gacitúa]]></surname>
<given-names><![CDATA[Ricardo]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
</contrib-group>
<aff id="Af1">
<institution><![CDATA[,Universidad de La Frontera Departamento de Ciencias de Computación e Informática ]]></institution>
<addr-line><![CDATA[Temuco ]]></addr-line>
<country>Chile</country>
</aff>
<pub-date pub-type="pub">
<day>00</day>
<month>06</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="epub">
<day>00</day>
<month>06</month>
<year>2023</year>
</pub-date>
<numero>50</numero>
<fpage>58</fpage>
<lpage>72</lpage>
<copyright-statement/>
<copyright-year/>
<self-uri xlink:href="http://www.scielo.pt/scielo.php?script=sci_arttext&amp;pid=S1646-98952023000200058&amp;lng=en&amp;nrm=iso"></self-uri><self-uri xlink:href="http://www.scielo.pt/scielo.php?script=sci_abstract&amp;pid=S1646-98952023000200058&amp;lng=en&amp;nrm=iso"></self-uri><self-uri xlink:href="http://www.scielo.pt/scielo.php?script=sci_pdf&amp;pid=S1646-98952023000200058&amp;lng=en&amp;nrm=iso"></self-uri><abstract abstract-type="short" xml:lang="es"><p><![CDATA[Resumen La corrección de las tareas de programación de computadoras es una actividad crítica y de alto consumo de tiempo para los académicos. Como respuesta a este problema, se han propuesto diversos sistemas de retroalimentación y calificación automática. Este artículo presenta una revisión de sistemas y enfoques de corrección y calificación automatizada de tareas de programación con el objetivo de ofrecer un mapa general de las funciones de estos sistemas y sus desafíos. Los resultados muestran que existe una importante diferencia entre la industria y la academia, así como una carencia en marcos de evaluación para estas herramientas.]]></p></abstract>
<abstract abstract-type="short" xml:lang="en"><p><![CDATA[Abstract: Reviewing programming assignments has become a time-consuming and critical activity for academics. Various feedback and automatic grading systems have been proposed in response to this problem. This article presents a review of various systems and automated feedback approaches for grading programming assignments to provide an overview of the functions of these systems and their challenges. The results show a difference between the tools proposed by the academy and those proposed by the industry, as well as a lack of evaluation frameworks.]]></p></abstract>
<kwd-group>
<kwd lng="es"><![CDATA[Programación]]></kwd>
<kwd lng="es"><![CDATA[enseñanza]]></kwd>
<kwd lng="es"><![CDATA[calificación automática]]></kwd>
<kwd lng="es"><![CDATA[retroalimentación]]></kwd>
<kwd lng="es"><![CDATA[resultados de aprendizaje]]></kwd>
<kwd lng="en"><![CDATA[Programming]]></kwd>
<kwd lng="en"><![CDATA[teaching]]></kwd>
<kwd lng="en"><![CDATA[automatic grading]]></kwd>
<kwd lng="en"><![CDATA[feedback]]></kwd>
<kwd lng="en"><![CDATA[learning outcomes]]></kwd>
</kwd-group>
</article-meta>
</front><back>
<ref-list>
<ref id="B1">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Aldriye]]></surname>
<given-names><![CDATA[H.]]></given-names>
</name>
<name>
<surname><![CDATA[Alkhalaf]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[Alkhalaf]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Automated grading systems for programming assignments: A literature review]]></article-title>
<source><![CDATA[International Journal of Advanced Computer Science and Applications]]></source>
<year>2019</year>
<volume>10</volume>
<numero>3</numero>
<issue>3</issue>
</nlm-citation>
</ref>
<ref id="B2">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Armstrong]]></surname>
<given-names><![CDATA[A. M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[The development of self-regulation skills through the modeling and structuring of computer programming]]></article-title>
<source><![CDATA[Educational Technology Research and Development]]></source>
<year>1989</year>
<volume>37</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>69-76</page-range></nlm-citation>
</ref>
<ref id="B3">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Bianchi]]></surname>
<given-names><![CDATA[I. S.]]></given-names>
</name>
<name>
<surname><![CDATA[Daehn]]></surname>
<given-names><![CDATA[C. M.]]></given-names>
</name>
<name>
<surname><![CDATA[Dávila]]></surname>
<given-names><![CDATA[G. A.]]></given-names>
</name>
<name>
<surname><![CDATA[Tovma]]></surname>
<given-names><![CDATA[N.]]></given-names>
</name>
<name>
<surname><![CDATA[Shurenov]]></surname>
<given-names><![CDATA[N.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Cursos online Abertos Massivos (MOOCs) como potencializadores do conhecimento em Instituições de Educação Superior]]></article-title>
<source><![CDATA[RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação]]></source>
<year>2022</year>
<numero>48</numero>
<issue>48</issue>
<page-range>59-73</page-range></nlm-citation>
</ref>
<ref id="B4">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Blinkin]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Dozens of U.S. colleges switching back to online classes as COVID-19 cases soar]]></article-title>
<source><![CDATA[Global news]]></source>
<year>2022</year>
</nlm-citation>
</ref>
<ref id="B5">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Budd]]></surname>
<given-names><![CDATA[T. A.]]></given-names>
</name>
<name>
<surname><![CDATA[Angluin]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Two notions of correctness and their relation to testing]]></article-title>
<source><![CDATA[Acta informatica]]></source>
<year>1982</year>
<volume>18</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>31-45</page-range></nlm-citation>
</ref>
<ref id="B6">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Carvallo]]></surname>
<given-names><![CDATA[J. P.]]></given-names>
</name>
<name>
<surname><![CDATA[Franch]]></surname>
<given-names><![CDATA[X.]]></given-names>
</name>
<name>
<surname><![CDATA[Quer]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
</person-group>
<source><![CDATA[Managing non-technical requirements in COTS components selection]]></source>
<year>2006</year>
<conf-name><![CDATA[ IEEE Int. Requirements Engineering Conf. (RE'06)]]></conf-name>
<conf-loc> </conf-loc>
<publisher-name><![CDATA[IEEE]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B7">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Costa]]></surname>
<given-names><![CDATA[J. M.]]></given-names>
</name>
<name>
<surname><![CDATA[Miranda]]></surname>
<given-names><![CDATA[G. L.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Desenvolvimento e validação de uma prova de avaliação das competências iniciais de programação]]></article-title>
<source><![CDATA[RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação]]></source>
<year>2017</year>
<numero>25</numero>
<issue>25</issue>
<page-range>66-81</page-range></nlm-citation>
</ref>
<ref id="B8">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Daradoumis]]></surname>
<given-names><![CDATA[T.]]></given-names>
</name>
<name>
<surname><![CDATA[Bassi]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
<name>
<surname><![CDATA[Xhafa]]></surname>
<given-names><![CDATA[F.]]></given-names>
</name>
<name>
<surname><![CDATA[Caballé]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
</person-group>
<source><![CDATA[A review on massive e-learning (MOOC) design, delivery and assessment]]></source>
<year>2013</year>
<conf-name><![CDATA[ Int. Conf. on P2P, Parallel, Grid, Cloud and Internet Computing]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B9">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Douce]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
<name>
<surname><![CDATA[Livingstone]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
<name>
<surname><![CDATA[Orwell]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Automatic test-based assessment of programming: A review]]></article-title>
<source><![CDATA[Journal on Educational Resources in Computing]]></source>
<year>2005</year>
<volume>5</volume>
<numero>3</numero>
<issue>3</issue>
<page-range>4</page-range></nlm-citation>
</ref>
<ref id="B10">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Fuller]]></surname>
<given-names><![CDATA[U.]]></given-names>
</name>
<name>
<surname><![CDATA[Johnson]]></surname>
<given-names><![CDATA[C. G.]]></given-names>
</name>
<name>
<surname><![CDATA[Ahoniemi]]></surname>
<given-names><![CDATA[T.]]></given-names>
</name>
<name>
<surname><![CDATA[Cukierman]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
<name>
<surname><![CDATA[Hernán Losada]]></surname>
<given-names><![CDATA[I.]]></given-names>
</name>
<name>
<surname><![CDATA[Jackova]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
</person-group>
<source><![CDATA[Developing a computer science specific learning taxonomy]]></source>
<year>2007</year>
<conf-name><![CDATA[ Conf. on Innovation and Technology in Computer Science Education]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B11">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Gupta]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
<name>
<surname><![CDATA[Anamika]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
</person-group>
<source><![CDATA[E-Assessment Tools for Programming Languages: A Review]]></source>
<year>2017</year>
<conf-name><![CDATA[ Conf. on Information Technology and Knowledge Management (ICITKM&#8217;2017)]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B12">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Hass]]></surname>
<given-names><![CDATA[B.]]></given-names>
</name>
<name>
<surname><![CDATA[Yuan]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
<name>
<surname><![CDATA[Li]]></surname>
<given-names><![CDATA[Z.]]></given-names>
</name>
</person-group>
<source><![CDATA[On the automatic assessment of learning outcomes in programming techniques]]></source>
<year>2019</year>
<conf-name><![CDATA[ Int. Conf. on Intelligent Systems and Knowledge Engineering (ISKE)]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B13">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Ismail]]></surname>
<given-names><![CDATA[M. H.]]></given-names>
</name>
<name>
<surname><![CDATA[Lakulu]]></surname>
<given-names><![CDATA[M. M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[A Critical Review on Recent Proposed Automated Programming Assessment Tool]]></article-title>
<source><![CDATA[Turkish Journal of Computer and Mathematics Education]]></source>
<year>2021</year>
<volume>12</volume>
<numero>3</numero>
<issue>3</issue>
<page-range>884-94</page-range></nlm-citation>
</ref>
<ref id="B14">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Ismail]]></surname>
<given-names><![CDATA[M. N.]]></given-names>
</name>
<name>
<surname><![CDATA[Ngah]]></surname>
<given-names><![CDATA[N. A.]]></given-names>
</name>
<name>
<surname><![CDATA[Umar]]></surname>
<given-names><![CDATA[I. N.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Instructional strategy in the teaching of computer programming: a need assessment analyses]]></article-title>
<source><![CDATA[The Turkish online Journal of Educational Technology]]></source>
<year>2010</year>
<volume>9</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>125-31</page-range></nlm-citation>
</ref>
<ref id="B15">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Maraza-Quispe]]></surname>
<given-names><![CDATA[B.]]></given-names>
</name>
<name>
<surname><![CDATA[Alejandro-Oviedo]]></surname>
<given-names><![CDATA[O.]]></given-names>
</name>
<name>
<surname><![CDATA[Fernández-Gambarini]]></surname>
<given-names><![CDATA[W.]]></given-names>
</name>
<name>
<surname><![CDATA[Cisneros-Chavez]]></surname>
<given-names><![CDATA[B.]]></given-names>
</name>
<name>
<surname><![CDATA[Choquehuanca-Quispe]]></surname>
<given-names><![CDATA[W.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Análisis de YouTube como herramienta de investigación documental en estudiantes de educación superior]]></article-title>
<source><![CDATA[Publicaciones]]></source>
<year>2020</year>
<volume>50</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>133-47</page-range></nlm-citation>
</ref>
<ref id="B16">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Nistala]]></surname>
<given-names><![CDATA[P.]]></given-names>
</name>
<name>
<surname><![CDATA[Nori]]></surname>
<given-names><![CDATA[K. V.]]></given-names>
</name>
<name>
<surname><![CDATA[Reddy]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
</person-group>
<source><![CDATA[Software quality models: A systematic mapping study]]></source>
<year>2019</year>
<conf-name><![CDATA[ IEEE/ACM Int. Conf. on Software and System Processes (ICSSP)]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B17">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Olechowski]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[Eppinger]]></surname>
<given-names><![CDATA[S. D.]]></given-names>
</name>
<name>
<surname><![CDATA[Joglekar]]></surname>
<given-names><![CDATA[N.]]></given-names>
</name>
</person-group>
<source><![CDATA[Technology readiness levels at 40: A study of state-of-the-art use, challenges, and opportunities]]></source>
<year>2015</year>
<conf-name><![CDATA[ Portland Int. Conf. on management of engineering and technology]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B18">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Poy]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
<name>
<surname><![CDATA[Gonzales-Aguilar]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Factores de éxito de los MOOC: algunas consideraciones críticas]]></article-title>
<source><![CDATA[RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação]]></source>
<year>2014</year>
<numero>E1</numero>
<issue>E1</issue>
<page-range>105-18</page-range></nlm-citation>
</ref>
<ref id="B19">
<nlm-citation citation-type="">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Sandín-Esteban]]></surname>
<given-names><![CDATA[M.P.]]></given-names>
</name>
</person-group>
<source><![CDATA[Investigación Cualitativa en Investigación en Educación: Fundamentos y Tradiciones]]></source>
<year>2003</year>
</nlm-citation>
</ref>
<ref id="B20">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Selby]]></surname>
<given-names><![CDATA[C. C.]]></given-names>
</name>
</person-group>
<source><![CDATA[Relationships: computational thinking, pedagogy of programming, and Bloom's Taxonomy]]></source>
<year>2015</year>
<conf-name><![CDATA[ Proc. of the workshop in primary and secondary computing education]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B21">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Souza]]></surname>
<given-names><![CDATA[D. M.]]></given-names>
</name>
<name>
<surname><![CDATA[Felizardo]]></surname>
<given-names><![CDATA[K. R.]]></given-names>
</name>
<name>
<surname><![CDATA[Barbosa]]></surname>
<given-names><![CDATA[E. F.]]></given-names>
</name>
</person-group>
<source><![CDATA[A systematic literature review of assessment tools for programming assignments]]></source>
<year>2016</year>
<conf-name><![CDATA[ Int. Conf. on Software Engineering Education and Training (CSEET)]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
<ref id="B22">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Striewe]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[Goedicke]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Effekte automatischer Bewertungen für Programmieraufgaben in Übungs-und Prüfungssituationen. DeLFI 2009-7]]></source>
<year>2009</year>
<publisher-name><![CDATA[Tagung der Fachgruppe E-Learning der Gesellschaft für Informatik eV]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B23">
<nlm-citation citation-type="confpro">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Storey]]></surname>
<given-names><![CDATA[M.A.]]></given-names>
</name>
<name>
<surname><![CDATA[Singer]]></surname>
<given-names><![CDATA[L.]]></given-names>
</name>
<name>
<surname><![CDATA[Cleary]]></surname>
<given-names><![CDATA[B.]]></given-names>
</name>
<name>
<surname><![CDATA[Figueira Filho]]></surname>
<given-names><![CDATA[F.]]></given-names>
</name>
<name>
<surname><![CDATA[Zagalsky]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
</person-group>
<source><![CDATA[The (r)evolution of social media in software engineering]]></source>
<year>2014</year>
<conf-name><![CDATA[ Future of Software Engineering (FOSE]]></conf-name>
<conf-loc> </conf-loc>
</nlm-citation>
</ref>
</ref-list>
</back>
</article>
