• Stefano Mariani
    Stefano Mariani, 11/03/2014 10:31

     REVIEW 1 

    PAPER: 28 TITLE: Integrating Agent Development Frameworks for Objective and Subjective Coordination: Experiments with TuCSoN and JADE AUTHORS: Stefano Mariani, Andrea Omicini and Luca Sangiorgi

    OVERALL EVALUATION: -2 (reject) REVIEWER'S CONFIDENCE: 4 (high) Originality: 3 (fair) Significance of contribution: 2 (poor) Quality of writing: 4 (good) Appropriateness for COORDINATION: 3 (fair)


     REVIEW 

    This paper proposes an integration of the coordination styles found in TuCSoN along with those found in Jade. The paper describes a motivation for doing this and basically the engineering involved in the integration.

    This paper has two main problems: the motivation is weak and there is a significant lack of innovation in the solution.

    Right off the bat, the paper completely neglects any work done in "swarm intelligence" in which lots (a high number) of autonomous components coordinate for some task. Both Jade and Tucson have been around for a long time; why is this the "right" time to do this work? Why is what this work is addressing a "real" problem? A better way to motivate the work described in the paper would be to provide a concrete example of an application-level coordination task that cannot be accomplished with today's state of the art but will be doable with the Tucson-Jade combo that the paper describes.

    The presentation of the contribution is confusing. The paper seems to neglect that, since Tucson, there have been several coordination models (and implemented systems) that provide non-blocking operation across tuple spaces. How does the proposed solution fit in that space?

    In Section 3, the paper gets to describing what has been done for this paper in combining Tucson and Jade. However, it's not clear what is really in this aside from making the agents both Jade agents and Tucson agents. What fundamental problem is the BridgeToTucson solving? Right now, it looks like a pretty ordinary and straightforward application of the Bridge pattern, which is a well known solution to this kind of problem.

    Some of the description of the functionality of the BridgeToTucson was a bit disconcerting. For instance, in the "clearTucsonOpResult" description, the paper says, "This method should be called as soon as the invoked operation result has been managed--at least, stored in a local variable." What are the ramifications of not doing this? Is it "dangerous"? Could it lead to race conditions between application agents? If so, then the potential for those race conditions exists. This definitely seems like something that should be delegated to the Bridge and handled automatically on behalf of the applications. Is there some key fundamental point I'm missing that allows this functionality to not be delegated?

    It's not clear what the point of the "showcase" is. It's not really an evaluation, and it doesn't argue that, even in the context of this application, the stated coordination tasks could not easily be performed without the BridgeToTucson solution (only that it is possible to perform them with the bridge).

    Another note: don't use colors to identify things since they don't show up in the common B&W version of the paper.


     REVIEW 2 

    PAPER: 28 TITLE: Integrating Agent Development Frameworks for Objective and Subjective Coordination: Experiments with TuCSoN and JADE AUTHORS: Stefano Mariani, Andrea Omicini and Luca Sangiorgi

    OVERALL EVALUATION: -2 (reject) REVIEWER'S CONFIDENCE: 3 (medium) Originality: 2 (poor) Significance of contribution: 2 (poor) Quality of writing: 4 (good) Appropriateness for COORDINATION: 4 (good)


     REVIEW 

    This paper presents an integration of the Tucson coordination model with the Jade framework. The aim is to allow Jade agents to access, via Jade services, Tucson tuple centres. More specifically, the paper proposes an extension of the integration of the two frameworks already introduced in a previous work (5). Such extension has been devised to solve the following problem of the previous integration: when a Tucson getter primitive (e.g., an "in" primitive) is executed, if no matching tuple is present in the tuple centre the operation is suspended (in fact "in" is a blocking primitive) and then the Jade agent is completely suspended too. This latter (undesired) effect is due to the fact that, although Jade supports the concurrent execution of the behaviours of an agent, Jade agents are implemented as a single-thread. This side-effect is removed in the proposed integration by means of a new component, namely Tucson4Jade bridge, that decouples the interactions between a Jade ! agent and a Tucson component. In this way, in case of a suspended operation the other activities of the Jade agent can continue their execution. 

    The paper is written in a good English and its topic well fits Coordination 2014. However, I do not think that the paper is mature enough for a conference publication. My main concerns about the paper are as follows:

    • The contribution of the work seems quite limited: it is just a technical solution to a specific interaction problem of a previous integration work. In fact, conceptually the solution relying on a "bridge" component is not particularly innovative. 
    • Section 3, which presents the main contribution of the work, is hard to read for readers not familiar with Jade, Tucson and their previous integration. Indeed, the presentation is too technical, and most concepts of the integration necessary for understanding the proposed approach are not described in the paper (for example, a description of the general architecture of the integration, as well as the corresponding illustrative figure mentioned in the paper, are missing and the reader is just referred to 5).
    • The paper mainly focusses on illustrating the technicalities of the proposed approach, while it is too weak for what concerns the motivations and the effectiveness of the integration of the two forms of coordination. It should be clarified (especially in the initial part of the paper) what are the benefits of the combined use of objective and subjective coordination models, by exploiting significative and realistic examples. It should be also clarified the motivation underlying the choices of Jade and Tucson among the various coordination frameworks. 
    • The discussion of related work should be significantly improved. Besides two papers about Jade, one about Linda, and one introducing subjective and objective coordination, all the other references are self-citations. In particular, no discussion about the integration of other coordination frameworks is provided, as well as about other attempts of integration of different coordination models.
      • Minor points **
    • Fig. 1: it is hard to read
    • Fig. 2 (and related comments in the text): why the first message is labelled by "2"? what is the meaning of labels "2.2.1", "2.2.2" and "2.2.2.1"?
    • Section 3: please, avoid to mention and discuss about a figure in another paper (Fig.1 of 5).
    • Page 8: please, take care of page margins.
    • Fig. 5 is too small. 
    • Fig. 12: it should appear at page 11, where it is illustrated, not at page 17.
    • Fig. 9 is too small. 
    • Conclusion: no discussion of ongoing and future work is provided.

     REVIEW 3 

    PAPER: 28 TITLE: Integrating Agent Development Frameworks for Objective and Subjective Coordination: Experiments with TuCSoN and JADE AUTHORS: Stefano Mariani, Andrea Omicini and Luca Sangiorgi

    OVERALL EVALUATION: -2 (reject) REVIEWER'S CONFIDENCE: 5 (expert) Originality: 2 (poor) Significance of contribution: 2 (poor) Quality of writing: 2 (poor) Appropriateness for COORDINATION: 4 (good)


     REVIEW 

    Although the subject of the paper would surely fit COORDINATION, in its present shape I see no real interesting contribution in the paper.

    The paper fails to give an actual description of the proposed solution that can be reused in other settings, and only presents an implementation using their tools TUCSON and JADE (such implementation fills most of the paper, and it's hard to follow).

    But most of all, the paper does not have a Related Work Section (!!!). It cites 13 papers and most of them are written by the authors (!!!). I find this very astonishing and very offensive.

    I'm sure the authors are aware of many other Java frameworks for coordination (in particular, with tuples in a distributed settings).


     REVIEW 4 

    PAPER: 28 TITLE: Integrating Agent Development Frameworks for Objective and Subjective Coordination: Experiments with TuCSoN and JADE AUTHORS: Stefano Mariani, Andrea Omicini and Luca Sangiorgi

    OVERALL EVALUATION: -2 (reject) REVIEWER'S CONFIDENCE: 4 (high) Originality: 3 (fair) Significance of contribution: 2 (poor) Quality of writing: 3 (fair) Appropriateness for COORDINATION: 3 (fair)


     REVIEW 

    The paper presents a technical solution for the integration of coordination media in a single-threaded cooperative multitasking runtime environment, in the form of an improvement over the integration of the TuCSoN coordination framework in the JADE multiagent system proposed in reference 5 (2004) and without regard for the multithreading support introduced in JADE 3.2 (July 2004).

    5 Omicini A. et al. (2004) Multi-agent infrastructure for objective and subjective coordination. Applied AI 18(9-10):815-831

    Re: Overall evaluation:

    While apparently a technically clean exercise, the submission has little to offer in terms of advancing the state of the art but could be viewed as a contribution illustrating the challenges and a possible solution for integration of coordination infrastructures into legacy and embedded systems platforms such as e.g. Arduino (we do acknowledge the existence of efforts for (pseudo)multithreading support also for this platform) -- see also "Appropriateness for COORDINATION". In this sense, it could be considered for a poster accept (possibly jointly with a demo).

    Re: Originality:

    The paper is concerned with ameliorating the impact of the integration of a dedicated coordination framework with a pseudo-multitasking (non-preemptive, round-robin multitasking) infrastructure through the relinquishing of the clear separation of concerns of the coordination framework in a manner that follows the example set by the cooperative multitasking solution, which already defers responsibilities to user programming.

    Setting out from the baseline given by previous work (5, see above), the contribution focuses on the refinement of the technical infrastructure and the presentation of the operational semantics supported by it. Essentially, it addresses the replacement of a blocking call by a callback mechanism; other than the technical aspects of the particular embedding in the JADE design framework, we do not see any advancement of the state of the art. 

    We also have to note that the authors silently ignore the introduction of multithreading support in JADE (shortly after the authoring of the based line paper/reference 5).

    Re: Significance of the contribution:

    The contribution does offer a critique of the earlier baseline implementation in 5; an in-depth evaluation of the proposed design and a balanced critique of it are however missing. Furthermore, the presented work is seriously lacking in consideration of the broader state of the art -- this is very clearly exposed by the fact that 9 out of the 12 references are self-references (i.e., include at least one co-author of the present paper and are papers highly related to/focusing on the subject matter at hand) -- the remaining 3 being standard references.

    Overall the bibliography is lacking in scope and is dated.

    Any larger significance of the paper as implied by the assertion in the Conclusions: "we highlighted where most of the integration efforts

        • for instance 5 - fail" thus is not demonstrated/corroborated
        • see also "Appropriateness for COORDINATION", below.

    Quality of writing

    The basic design and the structure of text does not match the present venue well - it goes into too much detail and fails to exploit theoretical vocabulary for a succinct and precise exposition and discussion of key matters. There is no section on related work nor a discussion section; the "conclusion" section does not go beyond a summary repetition of the content (that would be better suited than the current abstract).

    The style is at times overly verbose for a scientific contribution of limited length (with 17 pages, the submission exceeds the prescribed length limit).

    While the overall quality of English language is good, there are a number of recurrent annoyed issues that ought to be studied with due care and resolved/_mastered_ by the (first) author. These include:

    • Failure to use possessive case (i.e., " 's ")
    • Wrong use of "such" (as a first remedy, replace all occurrences by "this"; likewise, replace "keep on" by "continue")
    • Missing (definite) articles
    • Use of vacuous and colloquial expressions (that can mostly be omitted without loss: e.g., "In fact,", "Obviously,", "In particular", "Roughly speaking,", "Accordingly,"); circumscriptions ("Put in other words..."); colloquial and otherwise inappropriate (not verified?) vocabulary ("a truthful technological integration"; "we develop on the work done in 5"; to "successfully solve" an issue; to investigate something "deeply")
    • Excessive composite forms, e.g.:

    "an asynchronous operation invocation result" -> the result of the invocation of an asynchronous operation" "Obviously, the coordination operation actual invocation" -> The actual invocation of the coordination operation "the BridgeToTucson class is the actual responsible for TuCSoN services exploitation by JADE agents" -> "the BridgeToTucson class is (the one actually) responsible for the exploitation of TuCSoN services by JADE agents"

    We encountered one unintelligible sentence in subsection 3.1: "In fact, given that such behaviour differs from the caller one, and is implemented by Jade programmers, a mechanism for sharing TuCSoN completion events between custom-programmed" Jade behaviours should be devised out."

    Formal issues:

    • Key terminology (e.g. "TuCSoN" in section 1, "TuCSoN4JADE" in section 3), abbreviations and acronyms should always be explained/introduced upon first use.
    • It is not admissible to refer the reader to some other publication for information that is required in order to understand the present paper (--> cf. the repeated references to "Fig.1 of 5" in section 3!)
    • Figures should appear in the order referenced in the main text (-> Fig.12)
    • Use of/reliance on colour in figures and diagrams-this discriminates against colour-blind readers and hampers (e.g. hardcopy) reproduction in black and white; note also that resulting contrast is insufficient (see e.g. Fig.4)
    • URLs should come with the date of last visit
    • The references for quotations (including material such as figures and tables!) should always come with page numbers. 

    Appropriateness for COORDINATION:

    For the present purposes, it would have been preferable to place the focus on fundamental issues of such an integration and on the evaluation and balanced critiquing of the particular solution proposed, rather than getting lost in technical detail regarding a particular multiagent technologies setting

    Some detailed comments:

    Abstract:

    • The abstract does not provide a useful summary of the actual nature and scope of the contribution, it needs to be revised: consider e.g. moving text from the current "Conclusions" into the abstract.

    Introduction:

    "Whereas the first approach seems to better suit systems with a relatively small number of components exhibit a high degree of autonomy, the second fits well application scenarios involving much sic! more components with different degrees of autonomy.

    • Assertions should always be backed up by evidence/supportive arguments. Please provide e.g. some references for this claim 
    • In scientific writing, avoid unspecific qualitative expressions: what would "much more" (i.e.: *many* more) components amount to? (likewise, in a later sentence: what is the criterion for system to be "non-trivial"?)

    "unitary" -> unified

    "Accordingly, in this paper we take a step beyond what by proposing a straight TucsonjadeCoordination2014 integration allowing agents in a MAS to /simultaneously/ adopt both a subjective and an objective approach to coordination." -> By your own definition (given earlier), agents can adopt _subjective_ approaches only?

    A short introduction to/explanation of what TuCSoN is about is missing (e.g: the tuple-based coordination framework TuCSoN).

    Do find a different, more pertinent, expression for the current "finer" vs. "course" forms of integration of TuCSoN and JADE.

    Section 2:

    "In Section 3... whereas other aspects, although potentially relevant, are left out" => please do rephrase!

    The argument that "blocking coordination primitives harm agent autonomy" mixes up conceptual and implementational considerations. It should be reconsidered and rephrased. cf also the later statement: "Since TuCSoN agents can freely choose which version of the chosen coordination operation to invoke, their autonomy is preserved." -> it can be argued that rather than "preserving" help agent's autonomy, a concern that is not at the agent level is forced onto the agents (cf. the "autonomy" to select each single speech act vs. the engaging in a complete conversation protocol)

    "TuCSoN is a ... coordination model and infrastructure for open, distributed MAS." -> why only for MAS? (cf. the later remark: "incidentally, this helps in dealing with distributed scenarios TuCSoN is aimed at handling." <- which should be omitted: it is not of relevance here.)

    End of section: "ACCs are thus fundamental to guarantee and preserve agent autonomy 12 -- other than should read: "in addition to"..." -> it is difficult to see how ACCs could _guarantee_ an agent's autonomy?

    Section 3:

    Consider replacing the naming of asynchronous invocation of TuCSoN coordination operations "by interrupt" with "by callback"?

    in function calls, arguments are "passed", not "passed in"

    "Notice such asynchronous invocation is almost impossible to obtain.."

    • "almost impossible" -> "difficult"

    "On purpose, ..."

    • "On purpose" is not an English expression. Do you mean: "To this end" help

    Fig. 5: "two "helper" objects (which are TuCSoN agents)" -> which are JADE agents? (private JADE agents; JADE agents that are part of the TuCSoN4JADE infrastructure?)

    "Since the red path is usually taken first in a truthful run.."

    • What is a "truthful" run? Please rephrase.
    • In the next sentence, omit "(or better, its behaviour)" -- this is but _one_ example of spurious text that should be pruned.

    Section 4:

    • Please specify the version number of the Jade distribution employed

     and from where the example has been taken (e.g. the current distribution specifies the behaviours of the BookServerAgents as instances of CyclicBehavior, not "Behavior")

    "synthetically" represented

    • help - please rephrase

    "The crucial point is that by programming the OfferRequestsServer behaviour of BookSellerAgent using the already mentioned programming pat- tern depicted in Fig. 10 (as it was in "plain" Jade and as it is now in TuCSoN4Jade), such behaviour becomes seamlessly integrated with TuCSoN objective coordi- nation services, being it suspended and (automatically) resumed upon need- thanks to the cooperation between TuCSoN4Jade bridge and Jade platform. The same thing happens in PurchaseOrdersServer behaviour, whose code is sketched in Fig. 11, as well as in CFPHandler and Purchaser behaviours on the buyer side|not reported here for the lack of space."

    -> Our key point is that by using the programming pattern shown in Fig.8, behaviours such the BookSellerAgent's OfferRequestsServer (Fig.10) and the PurchaseOrdersServer (Fig.11) can be seamlessly integrated with the TuCSoN coordination services (and likewise the CFPHandler and Purchaser behaviours on the buyer's side).

    re: Fig.10: "CFP. The top listing is taken from Jade distribution, whereas the latter refers to the book trading example we reprogrammed using TuCSoN4Jade:"

    • The top listing includes "bridge.clearTucsonOpResult(this)" - this

     is clearly not taken from the Jade distribution.

    See elsewhere in this review re: missing discussion and lack of actual conclusions.

Partita IVA: 01131710376 - Copyright © 2008-2021 APICe@DISI Research Group - PRIVACY