Creating a sustainable residency research program is necessary to develop a sustainable research pipeline, as highlighted by the recent Society for Academic Emergency Medicine 2024 Consensus Conference. We sought to describe the implementation of a novel, immersive research program for first-year emergency medicine residents. We describe the curriculum development, rationale, implementation process, and lessons learned from the implementation of a year-long research curriculum for first-year residents. We further evaluated resident perception of confidence in research methodology, interest in research, and the importance of their research experience through a 32-item survey. In two cohorts, 25 first-year residents completed the program. All residents met their scholarly project requirements by the end of their first year. Two conference abstracts and one peer-reviewed publication were accepted for publication, and one is currently under review. Survey responses indicated that there was an increase in residents’ perceived confidence in research methodology, but this was limited by the small sample size. In summary, this novel resident research curriculum demonstrated a standardized, reproducible, and sustainable approach to provide residents with an immersive research program.

Novel prediction methods should always be compared to a baseline to determine their performance. Without this frame of reference, the performance score of a model is basically meaningless. What does it mean when a model achieves an $F_1$ of 0.8 on a test set? A proper baseline is, therefore, required to evaluate the ‘goodness’ of a performance score. Comparing results with the latest state-of-the-art model is usually insightful. However, being state-of-the-art is dynamic, as newer models are continuously developed. Contrary to an advanced model, it is also possible to use a simple dummy classifier. However, the latter model could be beaten too easily, making the comparison less valuable. Furthermore, most existing baselines are stochastic and need to be computed repeatedly to get a reliable expected performance, which could be computationally expensive. We present a universal baseline method for all binary classification models, named the Dutch Draw (DD). This approach weighs simple classifiers and determines the best classifier to use as a baseline. Theoretically, we derive the DD baseline for many commonly used evaluation measures and show that in most situations it reduces to (almost) always predicting either zero or one. Summarizing, the DD baseline is general, as it is applicable to any binary classification problem; simple, as it can be quickly determined without training or parameter tuning; and informative, as insightful conclusions can be drawn from the results. The DD baseline serves two purposes. First, it is a robust and universal baseline that enables comparisons across research papers. Second, it provides a sanity check during the prediction model’s development process. When a model does not outperform the DD baseline, it is a major warning sign.

The US biomedical research workforce suffers from systemic barriers causing insufficient diversity and perpetuating inequity. To inform programming enhancing graduate program access, we implemented a formative mixed-method study to identify needed supports for program applications and graduate program success. Overall, results indicate value in added supports for understanding application needs, network development, critical thinking, time management, and reading academic/scientific literature. We find selected differences for underrepresented minority (URM) students compared to others, including in the value of psychosocial supports. This work can inform broader efforts to enhance graduate school access and provides foundation for further understanding of URM students’ experiences.

The purpose of this paper is to answer the research question: How may methods of Benefit–Cost Analysis (BCA) influence policy decision-makers? The new research is a continuation of the previous research, and it is updated with new data and new examples. As it is well-known by BCA experts and practitioners, BCA is a relatively simple and widely used technique for deciding whether to make a change. BCA is very important in political and governmental decisions. Governments are policy creators, and Governments are usually formed of politicians. How much do experts inside Governments influence the policymakers? Do politicians allow enough experts to influence policymaking? The research regarding the work of the Croatian Government’s Office for Legislation and its Unit for the system and coordination of the Assessment of the Effects of Regulations (Unit) showed that the analysis of that Unit was not very good in some very important laws. These laws were changed very often because in practice was seen that the application of some laws was not good, and had caused some problems for the citizens, caused legal uncertainty for them, and in some cases had caused a lot of damage.

Organizations supporting translational research and translational science, including Clinical and Translational Science Award (CTSA) hubs, provide a diverse and often changing array of resources, support, and services to a myriad of researchers and research efforts. While a wide-ranging scope of programs is essential to the advancement of translational research and science, it also complicates a systematic and unified process for tracking activities, studying research processes, and examining impact. To overcome these challenges, the Duke University School of Medicine’s CTSA hub created a data platform, Translational Research Accomplishment Cataloguer (TRACER), that provides capacity to enhance strategic decision-making, impact assessment, and equitable resource distribution. This article reviews TRACER development processes, provides an overview of the TRACER platform, addresses challenges in the development process, and describes avenues for addressing or overcoming these challenges. TRACER development allowed our hub to conceptually identify key processes and goals within programs and linkages between programs, and it sets the stage for advancing evidence-based improvement across our hub. This platform development provides key insight into facilitators that can inform other initiatives seeking to collect and align organizational data for strategic decision-making and impact assessment. TRACER or similar platforms are additionally well positioned to advance the study of translational science.

With assisted dying becoming increasingly available to people suffering from somatic diseases, the question arises whether those suffering from mental illnesses should also have access. At the heart of this difficult and complex matter are values such as equality and parity of esteem. These issues require humane deliberation.

Human evaluation of machine translation is considered the “gold standard” for evaluation, but it remains a challenging task for which to define best practices. Recent work has focused on incorporating intersentential context into human evaluation, to better distinguish between high-performing machine translation systems and human translations. In this work, we examine several ways that such context influences evaluation and evaluation protocols. We take a close look at annotator variation through the lens of calibration sets and focus on the implications for context-aware evaluation protocols. We then demonstrate one way in which degraded target-side intersentential context can influence annotator scores of individual sentences, a finding that supports the context-aware approach to evaluation and which also has implications for best practices in evaluation protocols.

Structured processes to improve the quality and impact of clinical and translational research are a required element of the Clinical and Translational Sciences Awards (CTSA) program and are central to awardees’ strategic management efforts. Quality improvement is often assumed to be an ordinary consequence of evaluation programs, in which standardized metrics are tabulated and reported externally. Yet evaluation programs may not actually be very effective at driving quality improvement: required metrics may lack direct relevance; they lack incentive to improve on areas of relative strength; and the validity of inter-site comparability may be limited. In this article, we describe how we convened leaders at our CTSA hub in an iterative planning process to improve the quality of our CTSA program by intentionally focusing on how data collection activities can primarily advance continuous quality improvement (CQI) rather than strictly serve as evaluative tools. We describe our CQI process, which consists of three key components: (1) Logic models outlining goals and associated mechanisms; (2) relevant metrics to evaluate performance improvement opportunities; and (3) an interconnected and collaborative CQI framework that defines actions and timelines to enhance performance.

Many comics are aesthetically trashy: They are immediately grasped and easily available. Historically, this trashiness is lobbed as an aesthetic defect of many comics, a defect for both their production and their appreciation. To defend these comics, some point to non-aesthetic values, like sociality. I argue that there is aesthetic value to these comics, and that it lies precisely in their trashy characteristics: their immediacy and availability. Many comics have these characteristics because many comics are cartooned. The immediacy of cartooning is precisely what makes so many ordinary comics beautiful in an ordinary way.

Intentional language creation is a mainstay of the modern world, having gained widespread notoriety in popular television shows and films, and even finding a home in academia in the form of undergraduate courses on invented languages. In this paper, we argue that constructed languages deserve more careful consideration than they currently receive either inside or outside academia. We provide guidelines for developing evaluative criteria to be used with constructed languages of various types and ask readers, whether academics or not, to consider the role they play as audience and critics in the unfolding of a new art form: the art of language invention.

A negated statement consists of three main components: the negation cue, the negation scope, and the negation reference. The negation cue is the indicator of negation, while the negation scope defines the extent of the negation. The negation reference, which may or may not be within the negation scope, is the part of the statement being negated. Although there has been considerable research on the negation cue and scope, little attention has been given to identifying negation references outside the scope, even though they make up almost half of all negations. In this study, our goal is to identify out-of-scope references (OSRs) to restore the meaning of truncated negated statements identified by negation detection systems. To achieve this, we augment the largest available Spanish clinical dataset by adding annotations for OSRs. Additionally, we fine-tune five robust BERT-based models using transfer learning to address negation detection, uncertainty detection, and OSR identification and linking with their respective negation scopes. Our best model achieves state-of-the-art performance in negation detection while also establishing a competitive baseline for OSR identification (Macro F1 = 0.56) and linking (Macro F1 = 0.86). We support these findings with relevant statistics from the newly annotated dataset and an extensive review of existing literature.

The question of context in neural machine translation often focuses on topics related to document-level translation or intersentential context. However, there is a wide range of other aspects that can be considered under the umbrella of context. In this work, we survey ways that researchers have incorporated context into neural machine translation systems and the evaluation thereof. This includes building translation systems that operate at the paragraph level or the document level or ones that translate at the sentence level but incorporate information from other sentences. We also consider how issues like terminology consistency, anaphora, and world knowledge or external information can be considered as types of context relevant to the task of machine translation and its evaluation. Closely tied to these topics is the question of how to best evaluate machine translation output in a way that is sensitive to the contexts in which it appears. To this end, we discuss work on incorporating context into both human and automatic evaluations of machine translation quality. Furthermore, we also discuss recent experiments in the field as they relate to the use of large language models in translation and evaluation. We conclude with a view of the future of machine translation, where we expect to see issues of context continue to come to the forefront.

This paper examines the replicability of studies in design research triggered by the replication crisis in psychology. It highlights the importance of replicating studies to ensure the robustness of research results and examines whether the description in a publication is sufficient to replicate. Therefore, the publication of a reference study was analysed and a replication study was conducted. The design of the replication study appears similar to the reference study, but the results differ. Possible reasons for the differences and implications for replication studies are discussed.

The research paper delves into the importance of point cloud data obtained from 3D scanning technology ensuring quality control in industrial settings. It presents a new heuristic approach that utilizes the wavelet algorithm and other techniques to detect and characterize induced forming defects accurately. The proposed approach offers more flexibility, ease of use, and better results based on descriptive and prescriptive analyses from DRM. The results demonstrate that the wavelet algorithm was successful in identifying and characterizing forming defects in point cloud data.