Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex states in addition to learning a policy. As a result, deep RL typically suffers from slow learning speeds and often requires a prohibitively large amount of training time and data to reach reasonable performance, making it inapplicable to real-world settings where data are expensive. In this work, we improve data efficiency in deep RL by addressing one of the two learning goals, feature learning. We leverage supervised learning to pre-train on a small set of non-expert human demonstrations and empirically evaluate our approach using the asynchronous advantage actor-critic algorithms in the Atari domain. Our results show significant improvements in learning speed, even when the provided demonstration is noisy and of low quality.

Telecommunications infrastructures are a natural application domain for the distributed software agent paradigm. The authors clarify the potential application of software agent technology in legacy and future communications systems, and provide an overview of publicly available research on software agents used for communications management. The authors focus on the intelligent agent type of software agent, although the paper also reviews the reasons why mobile agents have made an impact in this domain. The author's objective is to describe some of the intricacies of using the software agent approach for the management of communications systems. The paper is in four main sections. The first section provides a brief introduction to software agent technology. The second section considers general problems of network management and the reasons why software agents may provide a suitable solution. The third section reviews some selected research on agents in a telecommunications management framework. The final section concludes the paper by discussing some of the problems encountered and some future directions for further research.

As the number of deployed multi-agent applications increases, further and better experience with the technology is gained, enabling a strong evaluation of the field from a more practical perspective. In particular, questions relating to how the theory of multi-agent systems impacts on practice, and how the practical development itself compares with other technologies, can be answered in the light of a heightened level of maturity. Given the tensions between theoreticians and practitioners in computing in general, let alone their spats in AI or multi-agent systems in particular, the discussion on agent systems and applications was both vigorous and enthusiastic

Recently, many question answering systems that derive answers from linked data repositories have been developed. The purpose of this survey is to identify the common features and approaches of the semantic question answering (SQA) systems, although many different and prototype systems have been designed. The SQA systems use a formal query language like SPARQL and knowledge of a specific vocabulary. This paper analyses different frameworks, architectures, or systems that perform SQA and classifies SQA systems based on different criteria.

The Defense Advanced Research Projects Agency (DARPA) and US Air Force Research Laboratory Planning Initiative (ARPI) has initiated a project to draw on the range of previous work in planning and activity ontologies to create a practically useful Shared Planning and Activity Representation (SPAR) for use in technology and applications projects within their communities. This article describes the previous work which has been used to create the initial SPAR representation. Key examples of the work drawn upon are published in this issue. The paper provides a comprehensive bibliography and related world wide web resources for work in the area of plan, process and activity representation. SPAR is now being subjected to refinement during several review cycles by a number of expert and user panels.

Nowadays, there are a lot of techniques and tools for addressing the ontology matching problem; however, the complex nature of this problem means that the existing solutions are unsatisfactory. This work intends to shed some light on a more flexible way of matching ontologies using ontology meta-matching. This emerging technique selects appropriate algorithms and their associated weights and thresholds in scenarios where accurate ontology matching is necessary. We think that an overview of the problem and an analysis of the existing state-of-the-art solutions will help researchers and practitioners to identify the most appropriate specific features and global strategies in order to build more accurate and dynamic systems following this paradigm.

The popularity of smart cars is increasing around the world as they offer a wide range of services and conveniences. These smart cars are equipped with a variety of sensors generating a large amount of data, many of which are critical. Besides, there are multiple parties involved in the lifespan of a smart car, such as manufacturers, car owners, government agencies, and third-party service providers who also generate data about the vehicle. In addition to managing and sharing data among these entities in a secure and privacy-friendly way which is a great challenge itself, there exists a trust deficit about some types of data as they remain under the custody of the car owner (e.g. satellite navigation and mileage data) and can easily be manipulated. In this article, we propose a blockchain-assisted architecture enabling the owner of a smart car to create an immutable record of every data, called the autobiography of a car, generated within its lifespan. We also explain how the trust about this record is guaranteed by the immutability characteristic of the blockchain. Furthermore, the article describes how the proposed architecture enables a secure and privacy-preserving mechanism for sharing of smart car data among different parties.

Self-organizing software systems are an increasingly attractive approach to highly distributed, decentralized, dynamic applications. In some domains (such as the Internet), the interaction of originally independent systems yields a self-organizing system de facto, and engineers must take these characteristics into account to manage them. This review surveys current work in this field and outlines its main themes, identifies challenges for future research, and addresses the continuity between software engineering in general and techniques appropriate for self-organizing systems.

Informal logic (IL) is an area of philosophy rich in models of communication and discourse with a heavy focus on argument and ‘dialogue games’. Computational dialectics is a maturing strand of research that is focused on implementing these dialogue games. The aim of this paper is to review research on applying IL dialogue games into human–computer dialogue design. We argue that IL dialogue games tend to have a number of attractive properties for human computer dialogue and that their computational utilization in this area has been increasing recently. Despite the strength of the case for IL, a number of important barriers need to be overcome if the potential of IL is to be fulfilled. These barriers are examined and means of overcoming them discussed.

This paper reviews a number of applications of machine learning to industrial control problems. We take the point of view of trying to automatically build rule-based reactive systems for tasks that, if performed by humans, would require a high degree of skill, yet are generally performed without thinking. Such skills are said to be sub-cognitive. While this might seem restrictive, most human skill is executed subconsciously and only becomes conscious when an unfamiliar circumstance is encountered. This kind of skill lends itself well to representation by a reactive system, that is, one that does not create a detailed model of the world, but rather, attempts to match percepts with actions in a very direct manner.

Automated machine classification will play a vital role in the machine learning and data mining. It is probable that each classifier will work well on some data sets and not so well in others, increasing the evaluation significance. The performance of the learning models will intensely rely on upon the characteristics of the data sets. The previous outcomes recommend that overlapping between classes and the presence of noise has the most grounded impact on the performance of learning algorithm. The class overlap problem is a critical problem in which data samples appear as valid instances of more than one class which may be responsible for the presence of noise in data sets.

The objective of this paper is to comprehend better the data used as a part of machine learning problems so as to learn issues and to analyze the instances that are profoundly covered by utilizing new proposed overlap measures. The proposed overlap measures are Nearest Enemy Ratio, SubConcept Ratio, Likelihood Ratio and Soft Margin Ratio. To perform this experiment, we have created 438 binary classification data sets from real-world problems and computed the value of 12 data complexity metrics to find highly overlapped data sets. After that we apply measures to identify the overlapped instances and four noise filters to find the noisy instances. From results, we found that 60–80% overlapped instances are noisy instances in data sets by using four noise filters. We found that class overlap is a principal contributor to introduce class noise in data sets.

In contrast to standard approaches based on agent communication languages (ACLs), environment-based coordination is emerging as an interesting alternative for structuring interactions in multiagent systems (MASs). In particular, the notion of coordination artifacts has been proposed as an engineering methodology to build runtime abstractions effectively providing collaborating agents with specifically designed coordination tasks.

In this paper, we study the semantics for the interaction of agents with coordination artifacts playing the same role of ACL semantics, that is, supporting semantic interoperability between agents developed by different parties through the connection between rationality and interaction. Our approach is rooted on the notion of operating instructions of coordination artifacts, which—as with a manual for a human exploiting a device—describe the interaction protocols the agent can follow as well as the mentalistic semantics of each single interaction. By tackling some of the most relevant issues raised in the context of ACL semantics, our framework allows intelligent, BDI-like agents to carry on complex interactions through coordination artifacts in a rational way.

This paper proposes that managing uncertainty is a control problem, a task for the control component of AI systems that decides what to do next. This view emphasizes the process of planning and executing sequences of actions that simultaneously satisfy domain goals and minimize uncertainty. The paper reviews AI systems that manage uncertainty by control. It is not an exhaustive survey, but rather illustrates issues in managing uncertainty with selected AI programs.

Multi-agent systems are complex systems in which multiple autonomous entities, called agents, cooperate in order to achieve a common or personal goal. These entities may be computer software, robots, and also humans. In fact, many multi-agent systems are intended to operate in cooperation with or as a service for humans. Typically, multi-agent systems are designed assuming perfectly rational, self-interested agents, according to the principles of classical game theory. Recently, such strong assumptions have been relaxed in various ways. One such way is explicitly including principles derived from human behavior. For instance, research in the field of behavioral economics shows that humans are not purely self-interested. In addition, they strongly care about fairness. Therefore, multi-agent systems that fail to take fairness into account, may not be sufficiently aligned with human expectations and may not reach intended goals. In this paper, we present an overview of work in the area of fairness in multi-agent systems. More precisely, we first look at the classical agent model, that is, rational decision making. We then provide an outline of descriptive models of fairness, that is, models that explain how and why humans reach fair decisions. Then, we look at prescriptive, computational models for achieving fairness in adaptive multi-agent systems. We show that results obtained by these models are compatible with experimental and analytical results obtained in the field of behavioral economics.

The toxicity prediction problem lies squarely at the interface of biology, chemistry and computational domains and will require the integrated application of knowledge and approaches from each domain for its solution. It is not a single modeling problem, but rather multiple levels of modeling compartments derived from the goals of toxicity risk assessment. These compartments include different categories and characteristics of toxicity (e.g., cancer vs. non-cancer, acute vs. delayed) and, therein, different levels of biofunctional organization and multiple mechanisms of toxicity extending to the level of individual chemical structures. A toxicity prediction model should strive to resolve the global toxicity prediction problem to local modeling compartments that reflect coherent biofunctional mechanisms and common modes of action while retaining some level of useful generalizations. This paper attempts to use these general concepts to frame the challenges and opportunities for application of Artificial Intelligence (AI) and computer-based methods to the goal of chemical toxicity prediction.

This commentary describes notable commercial applications of case-based reasoning, including systems that have been in continuous profitable use for over a decade. It is divided into sections on engineering applications, helpdesk applications and on-line case-based reasoning.

Shells and high-level programming language environments suffer from a number of shortcomings as knowledge engineering tools. We conclude that a variety of knowledge representation formalisms and a variety of controls regimes are needed. In addition guidelines should be provided about when to choose which knowledge representation formalism and which control regime. The guidelines should be based on properties of the task and the domain of the expert system. In order to arrive at these guidelines we first critically review some of the classifications of expert systems in the literature. We then give our own list of criteria. We test this list applying our criteria to a number of existing expert systems. As a caveat, we have not yet made a systematic attempt at correlating the criteria and different knowledge representations formalisms and control regimes, although we make some preliminary remarks throughout the paper.

This paper provides a synthesis of ontology research and one of its most significant application areas, the business discipline of Knowledge Management (KM). Despite the fact such a synergy is in great demand between consumers and suppliers of ontology technology, there is a lack of understanding on one side and a lack of realism on the other. Due to the lack of communication and understanding between ontology research and applied KM there is the danger that mainstream KM will remain neutral to ontologies, while research develops as a largely unguided exploration.

By adopting a structured knowledge-level approach, coordination knowledge can be ascribed to groups (societies) of system components (agents) as a whole, rather than to individuals, in order to effectively rationalise complex patterns of interaction within intelligent (multi-agent) systems. Be it either explicitly represented at the symbol-level or hard-coded within specific coordination algorithms, coordination knowledge is instrumented by a wide and heterogeneous variety of coordination models, abstractions and technologies. Coordination knowledge engineering is then about eliciting, modelling and instrumenting coordination knowledge in a principled and effective manner.

In this introductory article, we briefly review two well-known frameworks to conceptualise coordination, then we discuss different dimensions along which coordination models can be classified, and analyse their impact on the design of coordination mechanisms and their supporting coordination knowledge. Finally, we sketch our view on coordination knowledge engineering and introduce the different contributions to this special issue along this line.

PROMAS (Programming Multi-Agent Systems) is an AgentLink technical forum that aims to bring together the researchers and practitioners from both academia and industry to discuss the problems related to the development of multi-agent systems and to evaluate the existing proposals and results. The specific focus of this technical forum is on agent-oriented programming languages and tools that can effectively and efficiently support the implementation of multi-agent systems. This paper provides an overview of the main contributions and conclusions derived from the first two editions of PROMAS technical forum.