fond
Model Checking Contest @ Petri Nets 2015
Bruxelles, Belgium, June 23, 2015
Marcie compared to other tools («Known» models, ReachabilityCardinality)
Last Updated
August 19, 2015

Introduction

This page presents how Marcie do cope efficiently with the ReachabilityCardinality examination face to the other participating tools. In this page, we consider «Known» models.

The next sections will show chart comparing performances in termsof both memory and execution time.The x-axis corresponds to the challenging tool where the y-axes represents Marcie' performances. Thus, points below the diagonal of a chart denote comparisons favorables to the tool whileothers corresponds to situations where the challenging tool performs better.

You might also find plots out of the range that denote the case were at least one tool could not answer appropriately (error, time-out, could not compute or did not competed).

Marcie versus GreatSPN-Meddly

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for GreatSPN-Meddly, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to GreatSPN-Meddly are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie GreatSPN-Meddly Both tools   Marcie GreatSPN-Meddly
Computed OK 22 32 51   Smallest Memory Footprint
Do not compete 0 114 0 Times tool wins 22 83
Error detected 35 47 1   Shortest Execution Time
Cannot Compute + Time-out 155 19 140 Times tool wins 37 68


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus ITS-Tools

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for ITS-Tools, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to ITS-Tools are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie ITS-Tools Both tools   Marcie ITS-Tools
Computed OK 50 201 23   Smallest Memory Footprint
Do not compete 0 0 0 Times tool wins 51 223
Error detected 25 114 11   Shortest Execution Time
Cannot Compute + Time-out 249 9 46 Times tool wins 51 223


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus LoLA2.0

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for LoLA2.0, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to LoLA2.0 are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie LoLA2.0 Both tools   Marcie LoLA2.0
Computed OK 13 191 60   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 14 250
Error detected 34 14 2   Shortest Execution Time
Cannot Compute + Time-out 295 0 0 Times tool wins 18 246


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus LTSMin

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for LTSMin, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to LTSMin are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie LTSMin Both tools   Marcie LTSMin
Computed OK 61 137 12   Smallest Memory Footprint
Do not compete 0 0 0 Times tool wins 66 144
Error detected 18 97 18   Shortest Execution Time
Cannot Compute + Time-out 185 30 110 Times tool wins 61 149


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus TAPAAL(MC)

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for TAPAAL(MC), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to TAPAAL(MC) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie TAPAAL(MC) Both tools   Marcie TAPAAL(MC)
Computed OK 14 177 59   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 16 234
Error detected 36 4 0   Shortest Execution Time
Cannot Compute + Time-out 270 2 25 Times tool wins 28 222


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus TAPAAL(SEQ)

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for TAPAAL(SEQ), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to TAPAAL(SEQ) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie TAPAAL(SEQ) Both tools   Marcie TAPAAL(SEQ)
Computed OK 12 199 61   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 13 259
Error detected 36 2 0   Shortest Execution Time
Cannot Compute + Time-out 290 0 5 Times tool wins 30 242


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus TAPAAL-OTF(PAR)

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for TAPAAL-OTF(PAR), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to TAPAAL-OTF(PAR) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie TAPAAL-OTF(PAR) Both tools   Marcie TAPAAL-OTF(PAR)
Computed OK 29 19 44   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 37 55
Error detected 36 24 0   Shortest Execution Time
Cannot Compute + Time-out 135 20 160 Times tool wins 45 47


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

Marcie versus TAPAAL-OTF(SEQ)

Some statistics are displayed below, based on 808 runs (404 for Marcie and 404 for TAPAAL-OTF(SEQ), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing Marcie to TAPAAL-OTF(SEQ) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  Marcie TAPAAL-OTF(SEQ) Both tools   Marcie TAPAAL-OTF(SEQ)
Computed OK 18 84 55   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 18 139
Error detected 36 0 0   Shortest Execution Time
Cannot Compute + Time-out 174 7 121 Times tool wins 34 123


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart