Merge branch 'fix_missing_node_py_changes' of https://github.com/AntelopeIO/leap into fix_missing_node_py_changes

fix noisy `ignoring packed attribute` warning on `IR::ControlStructureImm`

CMakeLists.txt

@@ -14,7 +14,7 @@ set( CMAKE_CXX_EXTENSIONS ON )
 set( CXX_STANDARD_REQUIRED ON)
 
 set(VERSION_MAJOR 4)
-set(VERSION_MINOR 0)
+set(VERSION_MINOR 1)
 set(VERSION_PATCH 0)
 set(VERSION_SUFFIX dev)
 
@@ -77,29 +77,9 @@ if(ENABLE_OC AND CMAKE_SIZEOF_VOID_P EQUAL 8 AND NOT WIN32)
       list(APPEND EOSIO_WASM_RUNTIMES eos-vm-oc)
       # EOS VM OC requires LLVM, but move the check up here to a central location so that the EosioTester.cmakes
       # can be created with the exact version found
-      # find_package VERSION range is only available in 3.19 and newer.
-      if(${CMAKE_VERSION} VERSION_LESS "3.19")
-         find_package(LLVM REQUIRED CONFIG)
-      else()
-         # Preferring the latest LLVM version, search through major 11..7 and minor 9..0 in reverse order.
-         # Minor versions may break API. This manifests in the LLVM config files such that a range of `11...<12`
-         # will NOT find LLVM `11.1`. We loop over minor as well as major to resolve this.
-         set(max_minor 9)
-         set(major 11)
-         set(minor ${max_minor})
-         while(NOT LLVM_FOUND AND major GREATER_EQUAL 7)
-            math(EXPR end "${major} + 1")
-            find_package(LLVM ${major}.${minor}...<${end} CONFIG QUIET)
-            if(minor GREATER 0)
-               math(EXPR minor "${minor} - 1")
-            else()
-               math(EXPR major "${major} - 1")
-               set(minor ${max_minor})
-            endif()
-         endwhile()
-      endif()
+      find_package(LLVM REQUIRED CONFIG)
       if(LLVM_VERSION_MAJOR VERSION_LESS 7 OR LLVM_VERSION_MAJOR VERSION_GREATER_EQUAL 12)
-         message(FATAL_ERROR "Leap requires an LLVM version 7 through 11")
+	      message(FATAL_ERROR "Leap requires an LLVM version 7 through 11")
       endif()
    endif()
 endif()

README.md

@@ -72,7 +72,7 @@ git clone --recursive https://github.com/AntelopeIO/leap.git
 git clone --recursive git@github.com:AntelopeIO/leap.git
 ```
 
-> ℹ️ **HTTPS vs. SSH Clone** ℹ️
+> ℹ️ **HTTPS vs. SSH Clone** ℹ️  
 Both an HTTPS or SSH git clone will yield the same result - a folder named `leap` containing our source code. It doesn't matter which type you use.
 
 Navigate into that folder:
@@ -96,13 +96,13 @@ git submodule update --init --recursive
 ### Step 3 - Build
 Select build instructions below for a [pinned build](#pinned-build) (preferred) or an [unpinned build](#unpinned-build).
 
-> ℹ️ **Pinned vs. Unpinned Build** ℹ️
+> ℹ️ **Pinned vs. Unpinned Build** ℹ️  
 We have two types of builds for Leap: "pinned" and "unpinned." The only difference is that pinned builds use specific versions for some dependencies hand-picked by the Leap engineers - they are "pinned" to those versions. In contrast, unpinned builds use the default dependency versions available on the build system at the time. We recommend performing a "pinned" build to ensure the compiler and boost versions remain the same between builds of different Leap versions. Leap requires these versions to remain the same, otherwise its state might need to be recovered from a portable snapshot or the chain needs to be replayed.
 
-> ⚠️ **A Warning On Parallel Compilation Jobs (`-j` flag)** ⚠️
+> ⚠️ **A Warning On Parallel Compilation Jobs (`-j` flag)** ⚠️  
 When building C/C++ software, often the build is performed in parallel via a command such as `make -j "$(nproc)"` which uses all available CPU threads. However, be aware that some compilation units (`*.cpp` files) in Leap will consume nearly 4GB of memory. Failures due to memory exhaustion will typically, but not always, manifest as compiler crashes. Using all available CPU threads may also prevent you from doing other things on your computer during compilation. For these reasons, consider reducing this value.
 
-> 🐋 **Docker and `sudo`** 🐋
+> 🐋 **Docker and `sudo`** 🐋  
 If you are in an Ubuntu docker container, omit `sudo` from all commands because you run as `root` by default. Most other docker containers also exclude `sudo`, especially Debian-family containers. If your shell prompt is a hash tag (`#`), omit `sudo`.
 
 #### Pinned Build
@@ -111,12 +111,14 @@ Make sure you are in the root of the `leap` repo, then run the `install_depts.sh
 sudo scripts/install_deps.sh
 ```
 
-Next, run the pinned build script. You have to give it three arguments, in the following order:
-  - A temporary folder, for all dependencies that need to be built from source.
-  - A build folder, where the binaries you need to install will be built to.
-  - The number of jobs or CPU cores/threads to use (note the [jobs flag](#step-3---build) warning above).
+Next, run the pinned build script. You have to give it three arguments in the following order:
+1. A temporary folder, for all dependencies that need to be built from source.
+1. A build folder, where the binaries you need to install will be built to.
+1. The number of jobs or CPU cores/threads to use (note the [jobs flag](#step-3---build) warning above).
 
-The following command runs the `pinned_build.sh` script, specifies a `deps` and `build` folder in the root of the Leap repo for the first two arguments, then builds the packages using all of your computer's CPU threads (Note: you don't need `sudo` for this command):
+> 🔒 You do not need to run this script with `sudo` or as root.
+
+For example, the following command runs the `pinned_build.sh` script, specifies a `deps` and `build` folder in the root of the Leap repo for the first two arguments, then builds the packages using all of your computer's CPU threads:
 ```bash
 scripts/pinned_build.sh deps build "$(nproc)"
 ```
@@ -146,11 +148,9 @@ To build, make sure you are in the root of the `leap` repo, then run the followi
 ```bash
 mkdir -p build
 cd build
-cmake -DCMAKE_BUILD_TYPE=Release ..
+cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/usr/lib/llvm-11 ..
 make -j "$(nproc)" package
 ```
-
-If CMake fails to find a valid llvm configure file, it may be necessary to add `-DCMAKE_PREFIX_PATH=/usr/lib/llvm-11` to the cmake command.
 </details>
 
 <details> <summary>Ubuntu 18.04 Bionic</summary>

libraries/chain/block_log.cpp

@@ -1533,7 +1533,7 @@ namespace eosio { namespace chain {
       ilog("blocks.log and blocks.index agree on number of blocks");
 
       if (interval == 0) {
-         interval = std::max((log_bundle.log_index.num_blocks() + 7) >> 3, 1);
+         interval = std::max((log_bundle.log_index.num_blocks() + 7u) >> 3, 1u);
       }
       uint32_t expected_block_num = log_bundle.log_data.first_block_num();
 

libraries/chain/include/eosio/chain/log_index.hpp

@@ -32,7 +32,7 @@ class log_index {
    bool is_open() const { return file_.is_open(); }
 
    uint64_t back() { return nth_block_position(num_blocks()-1); }
-   int      num_blocks() const { return num_blocks_; }
+   unsigned num_blocks() const { return num_blocks_; }
    uint64_t nth_block_position(uint32_t n) {
       file_.seek(n*sizeof(uint64_t));
       uint64_t r;

libraries/wasm-jit/Include/IR/Operators.h

@@ -12,10 +12,11 @@ namespace IR
 	struct NoImm {};
 	struct MemoryImm {};
 
+	PACKED_STRUCT(
 	struct ControlStructureImm
 	{
 		ResultType resultType{};
-	};
+	});
 
 	struct BranchImm
 	{
@@ -675,4 +676,19 @@ namespace IR
 	};
 
 	IR_API const char* getOpcodeName(Opcode opcode);
-}
+}
+
+//paranoia for future platforms
+static_assert(sizeof(IR::OpcodeAndImm<IR::NoImm>) == 2);
+static_assert(sizeof(IR::OpcodeAndImm<IR::MemoryImm>) == 2);
+static_assert(sizeof(IR::OpcodeAndImm<IR::ControlStructureImm>) == 3);
+static_assert(sizeof(IR::OpcodeAndImm<IR::BranchImm>) == 6);
+static_assert(sizeof(IR::OpcodeAndImm<IR::BranchTableImm>) == 18);
+static_assert(sizeof(IR::OpcodeAndImm<IR::LiteralImm<I32>>) == 6);
+static_assert(sizeof(IR::OpcodeAndImm<IR::LiteralImm<I64>>) == 10);
+static_assert(sizeof(IR::OpcodeAndImm<IR::LiteralImm<F32>>) == 6);
+static_assert(sizeof(IR::OpcodeAndImm<IR::LiteralImm<F64>>) == 10);
+static_assert(sizeof(IR::OpcodeAndImm<IR::GetOrSetVariableImm<false>>) == 6);
+static_assert(sizeof(IR::OpcodeAndImm<IR::CallImm>) == 6);
+static_assert(sizeof(IR::OpcodeAndImm<IR::CallIndirectImm>) == 6);
+static_assert(sizeof(IR::OpcodeAndImm<IR::LoadOrStoreImm<0>>) == 10);

scripts/install_deps.sh

@@ -1,6 +1,27 @@
-#!/usr/bin/env bash
-
+#!/bin/bash
 apt-get update
 apt-get update --fix-missing
-DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
-apt-get -y install zip unzip libncurses5 wget git build-essential cmake curl libgmp-dev libssl-dev libzstd-dev time zlib1g-dev libtinfo-dev bzip2 libbz2-dev python3 python3-numpy file
+export DEBIAN_FRONTEND='noninteractive'
+export TZ='Etc/UTC'
+apt-get install -y \
+    build-essential \
+    bzip2 \
+    cmake \
+    curl \
+    file \
+    git \
+    libbz2-dev \
+    libcurl4-openssl-dev \
+    libgmp-dev \
+    libncurses5 \
+    libssl-dev \
+    libtinfo-dev \
+    libzstd-dev \
+    python3 \
+    python3-numpy \
+    time \
+    tzdata \
+    unzip \
+    wget \
+    zip \
+    zlib1g-dev

scripts/pinned_build.sh

@@ -1,151 +1,185 @@
-#!/usr/bin/env bash
+#!/bin/bash
+set -eo pipefail
 
 echo "Leap Pinned Build"
 
 if [[ "$(uname)" == "Linux" ]]; then
-   if [[ -e /etc/os-release ]]; then
-      # obtain NAME and other information
-      . /etc/os-release
-      if [[ ${NAME} != "Ubuntu" ]]; then
-         echo "Currently only supporting Ubuntu based builds. Proceed at your own risk."
-      fi
-   else
-       echo "Currently only supporting Ubuntu based builds. /etc/os-release not found. Your Linux distribution is not supported. Proceed at your own risk."
-   fi
+    if [[ -e /etc/os-release ]]; then
+        # obtain NAME and other information
+        . /etc/os-release
+        if [[ "${NAME}" != "Ubuntu" ]]; then
+            echo "Currently only supporting Ubuntu based builds. Proceed at your own risk."
+        fi
+    else
+        echo "Currently only supporting Ubuntu based builds. /etc/os-release not found. Your Linux distribution is not supported. Proceed at your own risk."
+    fi
 else
     echo "Currently only supporting Ubuntu based builds. Your architecture is not supported. Proceed at your own risk."
 fi
 
-if [ $# -eq 0 ] || [ -z "$1" ]
-   then
-      echo "Please supply a directory for the build dependencies to be placed and a directory for leap build and a value for the number of jobs to use for building."
-      echo "The binary packages will be created and placed into the leap build directory."
-      echo "./pinned_build.sh <dependencies directory> <leap build directory> <1-100>"
-      exit -1
+if [ $# -eq 0 ] || [ -z "$1" ]; then
+    echo "Please supply a directory for the build dependencies to be placed and a directory for leap build and a value for the number of jobs to use for building."
+    echo "The binary packages will be created and placed into the leap build directory."
+    echo "./pinned_build.sh <dependencies directory> <leap build directory> <1-100>"
+    exit 255
 fi
 
-CORE_SYM=EOS
+export CORE_SYM='EOS'
 # CMAKE_C_COMPILER requires absolute path
-DEP_DIR=`realpath $1`
-LEAP_DIR=$2
-JOBS=$3
+DEP_DIR="$(realpath "$1")"
+LEAP_DIR="$2"
+JOBS="$3"
 CLANG_VER=11.0.1
 BOOST_VER=1.70.0
 LLVM_VER=7.1.0
-ARCH=`uname -m`
 SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]:-$0}"; )" &> /dev/null && pwd 2> /dev/null; )";
 START_DIR="$(pwd)"
 
 
 pushdir() {
-   DIR=$1
-   mkdir -p ${DIR}
-   pushd ${DIR} &> /dev/null
+    DIR="$1"
+    mkdir -p "${DIR}"
+    pushd "${DIR}" &> /dev/null
 }
 
 popdir() {
-   EXPECTED=$1
-   D=`popd`
-   popd &> /dev/null
-   echo ${D}
-   D=`eval echo $D | head -n1 | cut -d " " -f1`
+    EXPECTED="$1"
+    D="$(popd)"
+    popd &> /dev/null
+    echo "${D}"
+    D="$(eval echo "$D" | head -n1 | cut -d " " -f1)"
 
-   # -ef compares absolute paths
-   if ! [[ ${D} -ef ${EXPECTED} ]]; then
-     echo "Directory is not where expected EXPECTED=${EXPECTED} at ${D}"
-     exit 1 
-   fi
+    # -ef compares absolute paths
+    if ! [[ "${D}" -ef "${EXPECTED}" ]]; then
+        echo "Directory is not where expected EXPECTED=${EXPECTED} at ${D}"
+        exit 1
+    fi
 }
 
 try(){
-   output=$($@)
-   res=$?
-   if [[ ${res} -ne 0 ]]; then
-      exit -1
-   fi
+    "$@"
+    res=$?
+    if [[ ${res} -ne 0 ]]; then
+        exit 255
+    fi
 }
 
 install_clang() {
-   CLANG_DIR=$1
-   if [ ! -d "${CLANG_DIR}" ]; then
-      echo "Installing Clang ${CLANG_VER} @ ${CLANG_DIR}"
-      mkdir -p ${CLANG_DIR}
-      CLANG_FN=clang+llvm-${CLANG_VER}-x86_64-linux-gnu-ubuntu-16.04.tar.xz
-      try wget -O ${CLANG_FN} https://github.com/llvm/llvm-project/releases/download/llvmorg-${CLANG_VER}/${CLANG_FN}
-      try tar -xvf ${CLANG_FN} -C ${CLANG_DIR}
-      pushdir ${CLANG_DIR}
-      mv clang+*/* .
-      popdir ${DEP_DIR}
-      rm ${CLANG_FN}
-   fi
-   export PATH=${CLANG_DIR}/bin:$PATH
-   export CLANG_DIR=${CLANG_DIR}
+    CLANG_DIR="$1"
+    if [ ! -d "${CLANG_DIR}" ]; then
+        echo "Installing Clang ${CLANG_VER} @ ${CLANG_DIR}"
+        mkdir -p "${CLANG_DIR}"
+        CLANG_FN="clang+llvm-${CLANG_VER}-x86_64-linux-gnu-ubuntu-16.04.tar.xz"
+        try wget -O "${CLANG_FN}" "https://github.com/llvm/llvm-project/releases/download/llvmorg-${CLANG_VER}/${CLANG_FN}"
+        try tar -xvf "${CLANG_FN}" -C "${CLANG_DIR}"
+        pushdir "${CLANG_DIR}"
+        mv clang+*/* .
+        popdir "${DEP_DIR}"
+        rm "${CLANG_FN}"
+    fi
+    export PATH="${CLANG_DIR}/bin:$PATH"
+    export CLANG_DIR="${CLANG_DIR}"
 }
 
 install_llvm() {
-   LLVM_DIR=$1
-   if [ ! -d "${LLVM_DIR}" ]; then
-      echo "Installing LLVM ${LLVM_VER} @ ${LLVM_DIR}"
-      mkdir -p ${LLVM_DIR}
-      try wget -O llvm-${LLVM_VER}.src.tar.xz https://github.com/llvm/llvm-project/releases/download/llvmorg-${LLVM_VER}/llvm-${LLVM_VER}.src.tar.xz
-      try tar -xvf llvm-${LLVM_VER}.src.tar.xz
-      pushdir "${LLVM_DIR}.src"
-      pushdir build
-      try cmake -DCMAKE_TOOLCHAIN_FILE=${SCRIPT_DIR}/pinned_toolchain.cmake -DCMAKE_INSTALL_PREFIX=${LLVM_DIR} -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=host -DLLVM_BUILD_TOOLS=Off -DLLVM_ENABLE_RTTI=On -DLLVM_ENABLE_TERMINFO=Off -DCMAKE_EXE_LINKER_FLAGS=-pthread -DCMAKE_SHARED_LINKER_FLAGS=-pthread -DLLVM_ENABLE_PIC=NO ..
-      try make -j${JOBS} 
-      try make -j${JOBS} install
-      popdir "${LLVM_DIR}.src"
-      popdir ${DEP_DIR}
-      rm -rf ${LLVM_DIR}.src 
-      rm llvm-${LLVM_VER}.src.tar.xz
-   fi
-   export LLVM_DIR=${LLVM_DIR}
+    LLVM_DIR="$1"
+    if [ ! -d "${LLVM_DIR}" ]; then
+        echo "Installing LLVM ${LLVM_VER} @ ${LLVM_DIR}"
+        mkdir -p "${LLVM_DIR}"
+        try wget -O "llvm-${LLVM_VER}.src.tar.xz" "https://github.com/llvm/llvm-project/releases/download/llvmorg-${LLVM_VER}/llvm-${LLVM_VER}.src.tar.xz"
+        try tar -xvf "llvm-${LLVM_VER}.src.tar.xz"
+        pushdir "${LLVM_DIR}.src"
+        pushdir build
+        try cmake -DCMAKE_TOOLCHAIN_FILE="${SCRIPT_DIR}/pinned_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="${LLVM_DIR}" -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=host -DLLVM_BUILD_TOOLS=Off -DLLVM_ENABLE_RTTI=On -DLLVM_ENABLE_TERMINFO=Off -DCMAKE_EXE_LINKER_FLAGS=-pthread -DCMAKE_SHARED_LINKER_FLAGS=-pthread -DLLVM_ENABLE_PIC=NO ..
+        try make -j "${JOBS}"
+        try make -j "${JOBS}" install
+        popdir "${LLVM_DIR}.src"
+        popdir "${DEP_DIR}"
+        rm -rf "${LLVM_DIR}.src"
+        rm "llvm-${LLVM_VER}.src.tar.xz"
+    fi
+    export LLVM_DIR="${LLVM_DIR}"
 }
 
 install_boost() {
-   BOOST_DIR=$1
+    BOOST_DIR="$1"
 
-   if [ ! -d "${BOOST_DIR}" ]; then
-      echo "Installing Boost ${BOOST_VER} @ ${BOOST_DIR}"
-      try wget -O boost_${BOOST_VER//\./_}.tar.gz https://boostorg.jfrog.io/artifactory/main/release/${BOOST_VER}/source/boost_${BOOST_VER//\./_}.tar.gz
-      try tar --transform="s:^boost_${BOOST_VER//\./_}:boost_${BOOST_VER//\./_}patched:" -xvzf boost_${BOOST_VER//\./_}.tar.gz -C ${DEP_DIR}
-      pushdir ${BOOST_DIR}
-      patch -p1 < "${SCRIPT_DIR}/0001-beast-fix-moved-from-executor.patch"
-      try ./bootstrap.sh -with-toolset=clang --prefix=${BOOST_DIR}/bin
-      ./b2 toolset=clang cxxflags='-stdlib=libc++ -D__STRICT_ANSI__ -nostdinc++ -I${CLANG_DIR}/include/c++/v1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fPIE' linkflags='-stdlib=libc++ -pie' link=static threading=multi --with-iostreams --with-date_time --with-filesystem --with-system --with-program_options --with-chrono --with-test -q -j${JOBS} install
-      popdir ${DEP_DIR}
-      rm boost_${BOOST_VER//\./_}.tar.gz
-   fi
-   export BOOST_DIR=${BOOST_DIR}
+    if [ ! -d "${BOOST_DIR}" ]; then
+        echo "Installing Boost ${BOOST_VER} @ ${BOOST_DIR}"
+        try wget -O "boost_${BOOST_VER//\./_}.tar.gz" "https://boostorg.jfrog.io/artifactory/main/release/${BOOST_VER}/source/boost_${BOOST_VER//\./_}.tar.gz"
+        try tar --transform="s:^boost_${BOOST_VER//\./_}:boost_${BOOST_VER//\./_}patched:" -xvzf "boost_${BOOST_VER//\./_}.tar.gz" -C "${DEP_DIR}"
+        pushdir "${BOOST_DIR}"
+        patch -p1 < "${SCRIPT_DIR}/0001-beast-fix-moved-from-executor.patch"
+        try ./bootstrap.sh -with-toolset=clang --prefix="${BOOST_DIR}/bin"
+        ./b2 toolset=clang cxxflags="-stdlib=libc++ -D__STRICT_ANSI__ -nostdinc++ -I\${CLANG_DIR}/include/c++/v1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fPIE" linkflags='-stdlib=libc++ -pie' link=static threading=multi --with-iostreams --with-date_time --with-filesystem --with-system --with-program_options --with-chrono --with-test -q -j "${JOBS}" install
+        popdir "${DEP_DIR}"
+        rm "boost_${BOOST_VER//\./_}.tar.gz"
+    fi
+    export BOOST_DIR="${BOOST_DIR}"
 }
 
-pushdir ${DEP_DIR}
+pushdir "${DEP_DIR}"
 
-install_clang ${DEP_DIR}/clang-${CLANG_VER}
-install_llvm ${DEP_DIR}/llvm-${LLVM_VER}
-install_boost ${DEP_DIR}/boost_${BOOST_VER//\./_}patched
+install_clang "${DEP_DIR}/clang-${CLANG_VER}"
+install_llvm "${DEP_DIR}/llvm-${LLVM_VER}"
+install_boost "${DEP_DIR}/boost_${BOOST_VER//\./_}patched"
 
 # go back to the directory where the script starts
-popdir ${START_DIR}
+popdir "${START_DIR}"
 
-pushdir ${LEAP_DIR}
+pushdir "${LEAP_DIR}"
 
 # build Leap
 echo "Building Leap ${SCRIPT_DIR}"
-try cmake -DCMAKE_TOOLCHAIN_FILE=${SCRIPT_DIR}/pinned_toolchain.cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=${LLVM_DIR}/lib/cmake -DCMAKE_PREFIX_PATH=${BOOST_DIR}/bin ${SCRIPT_DIR}/..
+try cmake -DCMAKE_TOOLCHAIN_FILE="${SCRIPT_DIR}/pinned_toolchain.cmake" -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="${LLVM_DIR}/lib/cmake" -DCMAKE_PREFIX_PATH="${BOOST_DIR}/bin" "${SCRIPT_DIR}/.."
 
-try make -j${JOBS}
+try make -j "${JOBS}"
 try cpack
 
-echo " .----------------.  .----------------.  .----------------.  .----------------. ";
-echo "| .--------------. || .--------------. || .--------------. || .--------------. |";
-echo "| |   _____      | || |  _________   | || |      __      | || |   ______     | |";
-echo "| |  |_   _|     | || | |_   ___  |  | || |     /  \     | || |  |_   __ \   | |";
-echo "| |    | |       | || |   | |_  \_|  | || |    / /\ \    | || |    | |__) |  | |";
-echo "| |    | |   _   | || |   |  _|  _   | || |   / ____ \   | || |    |  ___/   | |";
-echo "| |   _| |__/ |  | || |  _| |___/ |  | || | _/ /    \ \_ | || |   _| |_      | |";
-echo "| |  |________|  | || | |_________|  | || ||____|  |____|| || |  |_____|     | |";
-echo "| |              | || |              | || |              | || |              | |";
-echo "| '--------------' || '--------------' || '--------------' || '--------------' |";
-echo " '----------------'  '----------------'  '----------------'  '----------------' ";
-echo "Leap has successfully built and constructed its packages.  You should be able to find the packages at ${LEAP_DIR}.  Enjoy!!!"
+# art generated with DALL-E (https://openai.com/blog/dall-e), then fed through ASCIIart.club (https://asciiart.club) with permission
+cat <<'TXT' # BASH interpolation must remain disabled for the ASCII art to print correctly
+
+                                ,▄▄A`
+                             _╓▄██`
+                          ╓▄▓▓▀▀`
+                        ╓▓█▀╓▄▓
+                        ▓▌▓▓▓▀
+                      ,▄▓███▓H
+                    _╨╫▀╚▀╠▌`╙¥,
+                  ╓«    _╟▄▄   `½,                              ╓▄▄╦▄≥_
+                 ╙▓╫╬▒R▀▀╙▀▀▓φ_ «_╙Y╥▄mmMM#╦▄,_          ,╓╦mM╩╨╙╙╙\`║═
+                   ``       `▀▄__╫▓▓╨`    _```"""*ⁿⁿ^`````Ω,        `╟∩
+                              ╙▌▓▓"`    ,«ñ`            ╔╬▓▌⌂        ╔▌
+                               ╙█▌,,╔╗M╨,░             `  "╫▓m_      ╟H      _
+                        _,,,,__,╠█▓▒`  .╣▌µ _       _.╓╔▄▄▓█▓▓N_     ╙▀╩KKM╙╟▓N
+                      ,▄▓█▓████▀▀▀╙▀╓╔φ»█▓▓Ñ╦«, :»»µ╦▓▓█▀└╙▀███▓╥__   _,╓▄▓▓▓M▓`,
+                  __╓Φ▓█╫▓▓▓▓▓▓▓▓▓▓▓▓▓▀K▀▀███▓▓▓▓▓▓▀▀╙       `▀▀▀▓▄▄K╨╙└   `▀▌╙█▄*.
+                ,╓Φ▓▓▀▄▓▀`         ╙▀╙                                       ╙▓╙╙▓▄*
+              .▄▓╫▀╦▄▀`                                                       ╙▓╙µ╙▀
+             ▄▓▀╨▓▓╨_                                                          `▀▄▄M
+            _█▌▄▌╙`                                                               `╙
+             ╙└`
+
+
+                Ñ▓▓▓▓          ¢▄▄▄▄▄▄▄▄▄▄         ,           ,,,,,,,,,,,_
+               _╫▓▓█▌          ╟▓████████▀       ╓╣▓▌_         ╠╫▓█▓▓▓▓█▓▓▓▓▄
+               _▓▓▓█▌          ╟╫██▄,,,,_       æ▄███▓▄        ▐║████▀╨╨▀▓███▌
+               :╫▓▓█▌          ╟╢████████▓    ,╬███████▓,      ▐║████▓▄▄▓▓███▀
+               :╫▓▓█▌_        _╟║███▀▀▀▀▀▀   ╓╫███╣╫╬███▓N_    j▐██▀▀▀▀▀▀▀▀▀`
+    ___________]╫▓▓█▓φ╓╓╓╓╓╓,__╟╣█▌▌,,,,,_ _╬▓████████████▓▄_  ▐M█▌
+    _ _________]╫▌▓█████████▓▓▄╟╣████████▓▄╣██▓▀^    _ ╙▓██▓▓╗__M█▓
+      ___   _ _ ▀╣▀╣╩╩╩╩╩╩╩▀▀▀▀╙▀▀▀▀▀▀▀▀▀▀▀▀▀▀`          ╙▀▀▀▀╩═╩▀▀
+     _ __ __ _____ ___  ____ _  __  _  ____ ___
+    ____ ____ __  _ _  _ _ _ _ __   _ __ __
+    __ _ ________  ___ ________   ___ _ _ _      _
+       _ __  __     _ __ ____ _     _ ____ _   _ _       _
+     _  _ _     ____ ____ _ _    _ __  _ _ _ __
+
+---
+Leap has successfully built and constructed its packages. You should be able to
+find the packages at:
+TXT
+echo "${LEAP_DIR}"
+echo
+echo 'Thank you, fam!!'
+echo

tests/CMakeLists.txt

@@ -125,8 +125,12 @@ add_test(NAME nodeos_protocol_feature_test COMMAND tests/nodeos_protocol_feature
 set_property(TEST nodeos_protocol_feature_test PROPERTY LABELS nonparallelizable_tests)
 add_test(NAME compute_transaction_test COMMAND tests/compute_transaction_test.py -v -p 2 -n 3 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
 set_property(TEST compute_transaction_test PROPERTY LABELS nonparallelizable_tests)
-add_test(NAME read_only_trx_test COMMAND tests/read_only_trx_test.py -v -p 2 -n 3 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
-set_property(TEST read_only_trx_test PROPERTY LABELS nonparallelizable_tests)
+add_test(NAME read-only-trx-basic-test COMMAND tests/read_only_trx_test.py -v -p 2 -n 3 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
+set_property(TEST read-only-trx-basic-test PROPERTY LABELS nonparallelizable_tests)
+add_test(NAME read-only-trx-parallel-test COMMAND tests/read_only_trx_test.py -v -p 2 -n 3 --read-only-threads 3 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
+set_property(TEST read-only-trx-parallel-test PROPERTY LABELS nonparallelizable_tests)
+add_test(NAME read-only-trx-parallel-eos-vm-oc-test COMMAND tests/read_only_trx_test.py -v -p 2 -n 3 --eos-vm-oc-enable --read-only-threads 3 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
+set_property(TEST read-only-trx-parallel-eos-vm-oc-test PROPERTY LABELS nonparallelizable_tests)
 add_test(NAME subjective_billing_test COMMAND tests/subjective_billing_test.py -v -p 2 -n 4 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
 set_property(TEST subjective_billing_test PROPERTY LABELS nonparallelizable_tests)
 add_test(NAME get_account_test COMMAND tests/get_account_test.py -v -p 2 --clean-run ${UNSHARE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR})

tests/TestHarness/Cluster.py

@@ -1693,7 +1693,6 @@ class Cluster(object):
                             waitToComplete:bool=False, abiFile=None, actionsData=None, actionsAuths=None):
         Utils.Print("Configure txn generators")
         node=self.getNode(nodeId)
-        p2pListenPort = self.getNodeP2pPort(nodeId)
         info = node.getInfo()
         chainId = info['chain_id']
         lib_id = info['last_irreversible_block_id']
@@ -1702,13 +1701,12 @@ class Cluster(object):
         tpsLimitPerGenerator=tpsPerGenerator
 
         self.preExistingFirstTrxFiles = glob.glob(f"{Utils.DataDir}/first_trx_*.txt")
-
-        tpsTrxGensConfig = TpsTrxGensConfig(targetTps=targetTps, tpsLimitPerGenerator=tpsLimitPerGenerator)
+        connectionPairList = [f"{self.host}:{self.getNodeP2pPort(nodeId)}"]
+        tpsTrxGensConfig = TpsTrxGensConfig(targetTps=targetTps, tpsLimitPerGenerator=tpsLimitPerGenerator, connectionPairList=connectionPairList)
         self.trxGenLauncher = TransactionGeneratorsLauncher(chainId=chainId, lastIrreversibleBlockId=lib_id,
                                                     contractOwnerAccount=contractOwnerAcctName, accts=','.join(map(str, acctNamesList)),
                                                     privateKeys=','.join(map(str, acctPrivKeysList)), trxGenDurationSec=durationSec, logDir=Utils.DataDir,
-                                                    abiFile=abiFile, actionsData=actionsData, actionsAuths=actionsAuths,
-                                                    peerEndpoint=self.host, port=p2pListenPort, tpsTrxGensConfig=tpsTrxGensConfig)
+                                                    abiFile=abiFile, actionsData=actionsData, actionsAuths=actionsAuths, tpsTrxGensConfig=tpsTrxGensConfig)
 
         Utils.Print("Launch txn generators and start generating/sending transactions")
         self.trxGenLauncher.launch(waitToComplete=waitToComplete)
@@ -1729,4 +1727,7 @@ class Cluster(object):
                 for line in f:
                     firstTrxs.append(line.rstrip('\n'))
         Utils.Print(f"first transactions: {firstTrxs}")
-        node.waitForTransactionsInBlock(firstTrxs)
+        status = node.waitForTransactionsInBlock(firstTrxs)
+        if status is None:
+            Utils.Print('ERROR: Failed to spin up transaction generators: never received first transactions')
+        return status

tests/TestHarness/Node.py

@@ -484,6 +484,10 @@ class Node(Transactions):
 
     def scheduleSnapshot(self):
         return self.processUrllibRequest("producer", "schedule_snapshot")
+    
+    def scheduleSnapshotAt(self, sbn):
+        param = { "start_block_num": sbn, "end_block_num": sbn }
+        return self.processUrllibRequest("producer", "schedule_snapshot", param)
 
     # kill all existing nodeos in case lingering from previous test
     @staticmethod

tests/TestHarness/launch_transaction_generators.py

@@ -10,16 +10,19 @@ sys.path.append(harnessPath)
 
 from .testUtils import Utils
 from pathlib import Path
+from re import sub
 
 Print = Utils.Print
 
 class TpsTrxGensConfig:
 
-    def __init__(self, targetTps: int, tpsLimitPerGenerator: int):
+    def __init__(self, targetTps: int, tpsLimitPerGenerator: int, connectionPairList: list):
         self.targetTps: int = targetTps
         self.tpsLimitPerGenerator: int = tpsLimitPerGenerator
-
-        self.numGenerators = math.ceil(self.targetTps / self.tpsLimitPerGenerator)
+        self.connectionPairList = connectionPairList
+        self.numConnectionPairs = len(self.connectionPairList)
+        round_to_multiple = lambda num, multiple: math.ceil(num/multiple) * multiple
+        self.numGenerators = round_to_multiple(math.ceil(self.targetTps / self.tpsLimitPerGenerator), self.numConnectionPairs)
         self.initialTpsPerGenerator = math.floor(self.targetTps / self.numGenerators)
         self.modTps = self.targetTps % self.numGenerators
         self.cleanlyDivisible = self.modTps == 0
@@ -35,8 +38,7 @@ class TpsTrxGensConfig:
 class TransactionGeneratorsLauncher:
 
     def __init__(self, chainId: int, lastIrreversibleBlockId: int, contractOwnerAccount: str, accts: str, privateKeys: str, trxGenDurationSec: int, logDir: str,
-                 abiFile: Path, actionsData, actionsAuths,
-                 peerEndpoint: str, port: int, tpsTrxGensConfig: TpsTrxGensConfig):
+                 abiFile: Path, actionsData, actionsAuths, tpsTrxGensConfig: TpsTrxGensConfig):
         self.chainId = chainId
         self.lastIrreversibleBlockId = lastIrreversibleBlockId
         self.contractOwnerAccount  = contractOwnerAccount
@@ -46,14 +48,14 @@ class TransactionGeneratorsLauncher:
         self.tpsTrxGensConfig = tpsTrxGensConfig
         self.logDir = logDir
         self.abiFile = abiFile
-        self.actionsData=actionsData
-        self.actionsAuths=actionsAuths
-        self.peerEndpoint = peerEndpoint
-        self.port = port
+        self.actionsData = actionsData
+        self.actionsAuths = actionsAuths
 
     def launch(self, waitToComplete=True):
         self.subprocess_ret_codes = []
+        connectionPairIter = 0
         for id, targetTps in enumerate(self.tpsTrxGensConfig.targetTpsPerGenList):
+            connectionPair = self.tpsTrxGensConfig.connectionPairList[connectionPairIter].rsplit(":")
             popenStringList = [
                                 './tests/trx_generator/trx_generator',
                                 '--generator-id', f'{id}',
@@ -65,8 +67,8 @@ class TransactionGeneratorsLauncher:
                                 '--trx-gen-duration', f'{self.trxGenDurationSec}',
                                 '--target-tps', f'{targetTps}',
                                 '--log-dir', f'{self.logDir}',
-                                '--peer-endpoint', f'{self.peerEndpoint}',
-                                '--port', f'{self.port}']
+                                '--peer-endpoint', f'{connectionPair[0]}',
+                                '--port', f'{connectionPair[1]}']
             if self.abiFile is not None and self.actionsData is not None and self.actionsAuths is not None:
                 popenStringList.extend(['--abi-file', f'{self.abiFile}',
                                         '--actions-data', f'{self.actionsData}',
@@ -74,6 +76,7 @@ class TransactionGeneratorsLauncher:
             if Utils.Debug:
                 Print(f"Running trx_generator: {' '.join(popenStringList)}")
             self.subprocess_ret_codes.append(subprocess.Popen(popenStringList))
+            connectionPairIter = (connectionPairIter + 1) % len(self.tpsTrxGensConfig.connectionPairList)
         exitCodes=None
         if waitToComplete:
             exitCodes = [ret_code.wait() for ret_code in self.subprocess_ret_codes]
@@ -100,20 +103,20 @@ def parseArgs():
     parser.add_argument("abi_file", type=str, help="The path to the contract abi file to use for the supplied transaction action data")
     parser.add_argument("actions_data", type=str, help="The json actions data file or json actions data description string to use")
     parser.add_argument("actions_auths", type=str, help="The json actions auth file or json actions auths description string to use, containting authAcctName to activePrivateKey pairs.")
-    parser.add_argument("peer_endpoint", type=str, help="set the peer endpoint to send transactions to", default="127.0.0.1")
-    parser.add_argument("port", type=int, help="set the peer endpoint port to send transactions to", default=9876)
+    parser.add_argument("connection_pair_list", type=str, help="Comma separated list of endpoint:port combinations to send transactions to", default="localhost:9876")
     args = parser.parse_args()
     return args
 
 def main():
     args = parseArgs()
+    connectionPairList = sub('[\s+]', '', args.connection_pair_list)
+    connectionPairList = connectionPairList.rsplit(',')
 
     trxGenLauncher = TransactionGeneratorsLauncher(chainId=args.chain_id, lastIrreversibleBlockId=args.last_irreversible_block_id,
                                                    contractOwnerAccount=args.contract_owner_account, accts=args.accounts,
                                                    privateKeys=args.priv_keys, trxGenDurationSec=args.trx_gen_duration, logDir=args.log_dir,
                                                    abiFile=args.abi_file, actionsData=args.actions_data, actionsAuths=args.actions_auths,
-                                                   peerEndpoint=args.peer_endpoint, port=args.port,
-                                                   tpsTrxGensConfig=TpsTrxGensConfig(targetTps=args.target_tps, tpsLimitPerGenerator=args.tps_limit_per_generator))
+                                                   tpsTrxGensConfig=TpsTrxGensConfig(targetTps=args.target_tps, tpsLimitPerGenerator=args.tps_limit_per_generator, connectionPairList=connectionPairList))
 
 
     exit_codes = trxGenLauncher.launch()

tests/TestHarness/queries.py

@@ -118,10 +118,10 @@ class NodeosQueries:
         # could be a transaction response
         if cntxt.hasKey("processed"):
             cntxt.add("processed")
-            cntxt.add("action_traces")
-            cntxt.index(0)
             if not cntxt.isSectionNull("except"):
                 return "no_block"
+            cntxt.add("action_traces")
+            cntxt.index(0)
             return cntxt.add("block_num")
 
         # or what the trace api plugin returns
@@ -242,7 +242,7 @@ class NodeosQueries:
         assert(isinstance(transId, str))
         exitOnErrorForDelayed=not delayedRetry and exitOnError
         timeout=3
-        cmdDesc="get transaction_trace"
+        cmdDesc=self.fetchTransactionCommand()
         cmd="%s %s" % (cmdDesc, transId)
         msg="(transaction id=%s)" % (transId);
         for i in range(0,(int(60/timeout) - 1)):
@@ -295,8 +295,8 @@ class NodeosQueries:
         refBlockNum=None
         key=""
         try:
-            key="[transaction][transaction_header][ref_block_num]"
-            refBlockNum=trans["transaction_header"]["ref_block_num"]
+            key = self.fetchKeyCommand()
+            refBlockNum = self.fetchRefBlock(trans)
             refBlockNum=int(refBlockNum)+1
         except (TypeError, ValueError, KeyError) as _:
             Utils.Print("transaction%s not found. Transaction: %s" % (key, trans))
@@ -347,8 +347,8 @@ class NodeosQueries:
 
     def getTable(self, contract, scope, table, exitOnError=False):
         cmdDesc = "get table"
-        cmd="%s %s %s %s" % (cmdDesc, contract, scope, table)
-        msg="contract=%s, scope=%s, table=%s" % (contract, scope, table);
+        cmd=f"{cmdDesc} {self.cleosLimit} {contract} {scope} {table}"
+        msg=f"contract={contract}, scope={scope}, table={table}"
         return self.processCleosCmd(cmd, cmdDesc, exitOnError=exitOnError, exitMsg=msg)
 
     def getTableAccountBalance(self, contract, scope):
@@ -529,7 +529,7 @@ class NodeosQueries:
             return m.group(1)
         except subprocess.CalledProcessError as ex:
             end=time.perf_counter()
-            msg=ex.output.decode("utf-8")
+            msg=ex.stderr.decode("utf-8")
             Utils.Print("ERROR: Exception during code hash retrieval.  cmd Duration: %.3f sec.  %s" % (end-start, msg))
             return None
 
@@ -580,8 +580,9 @@ class NodeosQueries:
         except subprocess.CalledProcessError as ex:
             if not silentErrors:
                 end=time.perf_counter()
-                msg=ex.output.decode("utf-8")
-                errorMsg="Exception during \"%s\". Exception message: %s.  cmd Duration=%.3f sec. %s" % (cmdDesc, msg, end-start, exitMsg)
+                out=ex.output.decode("utf-8")
+                msg=ex.stderr.decode("utf-8")
+                errorMsg="Exception during \"%s\". Exception message: %s.  stdout: %s.  cmd Duration=%.3f sec. %s" % (cmdDesc, msg, out, end-start, exitMsg)
                 if exitOnError:
                     Utils.cmdError(errorMsg)
                     Utils.errorExit(errorMsg)

tests/TestHarness/transactions.py

@@ -107,7 +107,7 @@ class Transactions(NodeosQueries):
                 self.trackCmdTransaction(trans, reportStatus=reportStatus)
         except subprocess.CalledProcessError as ex:
             end=time.perf_counter()
-            msg=ex.output.decode("utf-8")
+            msg=ex.stderr.decode("utf-8")
             Utils.Print("ERROR: Exception during funds transfer.  cmd Duration: %.3f sec.  %s" % (end-start, msg))
             if exitOnError:
                 Utils.cmdError("could not transfer \"%s\" from %s to %s" % (amountStr, source, destination))
@@ -131,7 +131,7 @@ class Transactions(NodeosQueries):
                 Utils.Print("cmd Duration: %.3f sec" % (end-start))
         except subprocess.CalledProcessError as ex:
             end=time.perf_counter()
-            msg=ex.output.decode("utf-8")
+            msg=ex.stderr.decode("utf-8")
             Utils.Print("ERROR: Exception during spawn of funds transfer.  cmd Duration: %.3f sec.  %s" % (end-start, msg))
             if exitOnError:
                 Utils.cmdError("could not transfer \"%s\" from %s to %s" % (amountStr, source, destination))
@@ -158,15 +158,15 @@ class Transactions(NodeosQueries):
         except subprocess.CalledProcessError as ex:
             if not shouldFail:
                 end=time.perf_counter()
-                msg=ex.output.decode("utf-8")
-                Utils.Print("ERROR: Exception during set contract.  cmd Duration: %.3f sec.  %s" % (end-start, msg))
+                out=ex.output.decode("utf-8")
+                msg=ex.stderr.decode("utf-8")
+                Utils.Print("ERROR: Exception during set contract. stderr: %s.  stdout: %s.  cmd Duration: %.3f sec." % (msg, out, end-start))
                 return None
             else:
                 retMap={}
                 retMap["returncode"]=ex.returncode
                 retMap["cmd"]=ex.cmd
                 retMap["output"]=ex.output
-                retMap["stdout"]=ex.stdout
                 retMap["stderr"]=ex.stderr
                 return retMap
 
@@ -213,7 +213,7 @@ class Transactions(NodeosQueries):
                 Utils.Print("cmd Duration: %.3f sec" % (end-start))
             return (NodeosQueries.getTransStatus(retTrans) == 'executed', retTrans)
         except subprocess.CalledProcessError as ex:
-            msg=ex.output.decode("utf-8")
+            msg=ex.stderr.decode("utf-8")
             if not silentErrors:
                 end=time.perf_counter()
                 Utils.Print("ERROR: Exception during push transaction.  cmd Duration=%.3f sec.  %s" % (end - start, msg))
@@ -245,7 +245,6 @@ class Transactions(NodeosQueries):
         except subprocess.CalledProcessError as ex:
             msg=ex.stderr.decode("utf-8")
             output=ex.output.decode("utf-8")
-            msg=ex.output.decode("utf-8")
             if not silentErrors:
                 end=time.perf_counter()
                 Utils.Print("ERROR: Exception during push message. stderr: %s. stdout: %s.  cmd Duration=%.3f sec." % (msg, output, end - start))

tests/nodeos_snapshot_diff_test.py

@@ -126,13 +126,15 @@ try:
     waitForBlock(node0, blockNum, blockType=BlockType.lib)
 
     Print("Configure and launch txn generators")
+    
     targetTpsPerGenerator = 10
     testTrxGenDurationSec=60*30
     cluster.launchTrxGenerators(contractOwnerAcctName=cluster.eosioAccount.name, acctNamesList=[account1Name, account2Name],
                                 acctPrivKeysList=[account1PrivKey,account2PrivKey], nodeId=snapshotNodeId, tpsPerGenerator=targetTpsPerGenerator,
                                 numGenerators=trxGeneratorCnt, durationSec=testTrxGenDurationSec, waitToComplete=False)
 
-    cluster.waitForTrxGeneratorsSpinup(nodeId=snapshotNodeId, numGenerators=trxGeneratorCnt)
+    status = cluster.waitForTrxGeneratorsSpinup(nodeId=snapshotNodeId, numGenerators=trxGeneratorCnt)
+    assert status is not None, "ERROR: Failed to spinup Transaction Generators"
 
     blockNum=node0.getBlockNum(BlockType.head)
     timePerBlock=500
@@ -156,11 +158,8 @@ try:
     minReqPctLeeway=0.60
     minRequiredTransactions=minReqPctLeeway*transactionsPerBlock
     assert steadyStateAvg>=minRequiredTransactions, "Expected to at least receive %s transactions per block, but only getting %s" % (minRequiredTransactions, steadyStateAvg)
-
-    Print("Create snapshot")
-    ret = nodeProg.scheduleSnapshot()
-    assert ret is not None, "Snapshot creation failed"
     
+    Print("Create snapshot (node 0)")
     ret = nodeSnap.createSnapshot()
     assert ret is not None, "Snapshot creation failed"
     ret_head_block_num = ret["payload"]["head_block_num"]
@@ -177,6 +176,29 @@ try:
     Utils.processLeapUtilCmd("snapshot to-json --input-file {}".format(snapshotFile), "snapshot to-json", silentErrors=False)
     snapshotFile = snapshotFile + ".json"
 
+    Print("Trim programmable blocklog to snapshot head block num and relaunch programmable node")
+    nodeProg.kill(signal.SIGTERM)
+    output=cluster.getBlockLog(progNodeId, blockLogAction=BlockLogAction.trim, first=0, last=ret_head_block_num, throwException=True)
+    removeState(progNodeId)
+    Utils.rmFromFile(Utils.getNodeConfigDir(progNodeId, "config.ini"), "p2p-peer-address")
+    isRelaunchSuccess = nodeProg.relaunch(chainArg="--replay", addSwapFlags={}, timeout=relaunchTimeout, cachePopen=True)
+    assert isRelaunchSuccess, "Failed to relaunch programmable node"
+
+    Print("Schedule snapshot (node 2)")
+    ret = nodeProg.scheduleSnapshotAt(ret_head_block_num)
+    assert ret is not None, "Snapshot scheduling failed"
+
+    Print("Wait for programmable node lib to advance")
+    waitForBlock(nodeProg, ret_head_block_num+1, blockType=BlockType.lib)
+
+    Print("Kill programmable node")
+    nodeProg.kill(signal.SIGTERM)
+
+    Print("Convert snapshot to JSON")
+    progSnapshotFile = getLatestSnapshot(progNodeId)
+    Utils.processLeapUtilCmd("snapshot to-json --input-file {}".format(progSnapshotFile), "snapshot to-json", silentErrors=False)
+    progSnapshotFile = progSnapshotFile + ".json"
+
     Print("Trim irreversible blocklog to snapshot head block num")
     nodeIrr.kill(signal.SIGTERM)
     output=cluster.getBlockLog(irrNodeId, blockLogAction=BlockLogAction.trim, first=0, last=ret_head_block_num, throwException=True)
@@ -204,15 +226,6 @@ try:
     irrSnapshotFile = irrSnapshotFile + ".json"
 
     assert Utils.compareFiles(snapshotFile, irrSnapshotFile), f"Snapshot files differ {snapshotFile} != {irrSnapshotFile}"
-  
-    Print("Kill programmable node")
-    nodeProg.kill(signal.SIGTERM)
-
-    Print("Convert snapshot to JSON")
-    progSnapshotFile = getLatestSnapshot(progNodeId)
-    Utils.processLeapUtilCmd("snapshot to-json --input-file {}".format(progSnapshotFile), "snapshot to-json", silentErrors=False)
-    progSnapshotFile = progSnapshotFile + ".json"
-
     assert Utils.compareFiles(progSnapshotFile, irrSnapshotFile), f"Snapshot files differ {progSnapshotFile} != {irrSnapshotFile}"
 
     testSuccessful=True

tests/nodeos_startup_catchup.py

@@ -118,7 +118,8 @@ try:
                                 tpsPerGenerator=targetTpsPerGenerator, numGenerators=trxGeneratorCnt, durationSec=testTrxGenDurationSec,
                                 waitToComplete=False)
 
-    cluster.waitForTrxGeneratorsSpinup(nodeId=node0.nodeId, numGenerators=trxGeneratorCnt)
+    status = cluster.waitForTrxGeneratorsSpinup(nodeId=node0.nodeId, numGenerators=trxGeneratorCnt)
+    assert status is not None, "ERROR: Failed to spinup Transaction Generators"
 
     blockNum=head(node0)
     timePerBlock=500

tests/performance_tests/README.md

@@ -488,8 +488,7 @@ Performance Test Basic Single Test:
 * `abi_file`                    The path to the contract abi file to use for the supplied transaction action data
 * `actions_data`                The json actions data file or json actions data description string to use
 * `actions_auths`               The json actions auth file or json actions auths description string to use, containting authAcctName to activePrivateKey pairs.
-* `peer_endpoint`               set the peer endpoint to send transactions to, default="127.0.0.1"
-* `port`                        set the peer endpoint port to send transactions to, default=9876
+* `connection_pair_list`        Comma separated list of endpoint:port combinations to send transactions to
 </details>
 
 #### Transaction Generator

tests/performance_tests/performance_test_basic.py

@@ -327,7 +327,9 @@ class PerformanceTestBasic:
     def runTpsTest(self) -> PtbTpsTestResult:
         completedRun = False
         self.producerNode = self.cluster.getNode(self.producerNodeId)
-        self.producerP2pPort = self.cluster.getNodeP2pPort(self.producerNodeId)
+        self.connectionPairList = []
+        for producer in range(0, self.clusterConfig.pnodes):
+            self.connectionPairList.append(f"{self.cluster.getNode(producer).host}:{self.cluster.getNodeP2pPort(producer)}")
         self.validationNode = self.cluster.getNode(self.validationNodeId)
         self.wallet = self.walletMgr.create('default')
         self.setupContract()
@@ -371,13 +373,13 @@ class PerformanceTestBasic:
         self.cluster.biosNode.kill(signal.SIGTERM)
 
         self.data.startBlock = self.waitForEmptyBlocks(self.validationNode, self.emptyBlockGoal)
-        tpsTrxGensConfig = TpsTrxGensConfig(targetTps=self.ptbConfig.targetTps, tpsLimitPerGenerator=self.ptbConfig.tpsLimitPerGenerator)
+        tpsTrxGensConfig = TpsTrxGensConfig(targetTps=self.ptbConfig.targetTps, tpsLimitPerGenerator=self.ptbConfig.tpsLimitPerGenerator, connectionPairList=self.connectionPairList)
 
         self.cluster.trxGenLauncher = TransactionGeneratorsLauncher(chainId=chainId, lastIrreversibleBlockId=lib_id, contractOwnerAccount=self.clusterConfig.specifiedContract.account.name,
                                                        accts=','.join(map(str, self.accountNames)), privateKeys=','.join(map(str, self.accountPrivKeys)),
                                                        trxGenDurationSec=self.ptbConfig.testTrxGenDurationSec, logDir=self.trxGenLogDirPath,
                                                        abiFile=abiFile, actionsData=actionsDataJson, actionsAuths=actionsAuthsJson,
-                                                       peerEndpoint=self.producerNode.host, port=self.producerP2pPort, tpsTrxGensConfig=tpsTrxGensConfig)
+                                                       tpsTrxGensConfig=tpsTrxGensConfig)
 
         trxGenExitCodes = self.cluster.trxGenLauncher.launch()
         print(f"Transaction Generator exit codes: {trxGenExitCodes}")

tests/read_only_trx_test.py

@@ -1,8 +1,11 @@
 #!/usr/bin/env python3
 
 import random
+import time
+import signal
+import threading
 
-from TestHarness import Account, Cluster, TestHelper, Utils, WalletMgr
+from TestHarness import Account, Cluster, ReturnType, TestHelper, Utils, WalletMgr
 from TestHarness.TestHelper import AppArgs
 
 ###############################################################
@@ -29,6 +32,10 @@ pnodes=args.p
 topo=args.s
 delay=args.d
 total_nodes = pnodes if args.n < pnodes else args.n
+# For this test, we need at least 1 non-producer
+if total_nodes <= pnodes:
+    Print ("non-producing nodes %d must be greater than 0. Force it to %d. producing nodes: %d," % (total_nodes - pnodes, pnodes + 1, pnodes))
+    total_nodes = pnodes + 1
 debug=args.v
 nodesFile=args.nodes_file
 dontLaunch=nodesFile is not None
@@ -45,6 +52,7 @@ if nodesFile is not None:
 
 Utils.Debug=debug
 testSuccessful=False
+errorInThread=False
 
 random.seed(seed) # Use a fixed seed for repeatability.
 cluster=Cluster(walletd=True,unshared=args.unshared)
@@ -54,6 +62,9 @@ EOSIO_ACCT_PRIVATE_DEFAULT_KEY = "5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79z
 EOSIO_ACCT_PUBLIC_DEFAULT_KEY = "EOS6MRyAjQq8ud7hVNYcfnVPJqcVpscN5So8BhtHuGYqET5GDW5CV"
 
 try:
+    TestHelper.printSystemInfo("BEGIN")
+    cluster.setWalletMgr(walletMgr)
+
     if dontLaunch: # run test against remote cluster
         jsonStr=None
         with open(nodesFile, "r") as f:
@@ -71,11 +82,25 @@ try:
             cluster.killall(allInstances=killAll)
             cluster.cleanup()
 
-    Print ("producing nodes: %s, non-producing nodes: %d, topology: %s, delay between nodes launch(seconds): %d" % (pnodes, total_nodes-pnodes, topo, delay))
+    Print ("producing nodes: %d, non-producing nodes: %d, topology: %s, delay between nodes launch(seconds): %d" % (pnodes, total_nodes-pnodes, topo, delay))
 
+    cluster.killall(allInstances=killAll)
+    cluster.cleanup()
     Print("Stand up cluster")
+    # set up read-only options for API node
+    specificExtraNodeosArgs={}
+    # producer nodes will be mapped to 0 through pnodes-1, so the number pnodes is the no-producing API node
+    specificExtraNodeosArgs[pnodes]=" --plugin eosio::net_api_plugin"
+    if args.read_only_threads > 0:
+        specificExtraNodeosArgs[pnodes]+=" --read-only-threads "
+        specificExtraNodeosArgs[pnodes]+=str(args.read_only_threads)
+    if args.eos_vm_oc_enable:
+        specificExtraNodeosArgs[pnodes]+=" --eos-vm-oc-enable"
+    if args.wasm_runtime:
+        specificExtraNodeosArgs[pnodes]+=" --wasm-runtime "
+        specificExtraNodeosArgs[pnodes]+=args.wasm_runtime
     extraNodeosArgs=" --http-max-response-time-ms 990000 --disable-subjective-api-billing false "
-    if cluster.launch(pnodes=pnodes, totalNodes=total_nodes, topo=topo, delay=delay,extraNodeosArgs=extraNodeosArgs ) is False:
+    if cluster.launch(pnodes=pnodes, totalNodes=total_nodes, topo=topo, delay=delay, specificExtraNodeosArgs=specificExtraNodeosArgs, extraNodeosArgs=extraNodeosArgs ) is False:
         errorExit("Failed to stand up eos cluster.")
 
     Print ("Wait for Cluster stabilization")
@@ -97,7 +122,7 @@ try:
     userAccount = Account(userAccountName)
     userAccount.ownerPublicKey = EOSIO_ACCT_PUBLIC_DEFAULT_KEY
     userAccount.activePublicKey = EOSIO_ACCT_PUBLIC_DEFAULT_KEY
-    cluster.createAccountAndVerify(userAccount, cluster.eosioAccount)
+    cluster.createAccountAndVerify(userAccount, cluster.eosioAccount, stakeCPU=2000)
 
     noAuthTableContractDir="unittests/test-contracts/no_auth_table"
     noAuthTableWasmFile="no_auth_table.wasm"
@@ -116,28 +141,148 @@ try:
         }
         return apiNode.pushTransaction(trx, opts)
 
-    Print("Insert a user")
-    results = sendTransaction('insert', {"user": userAccountName, "id": 1, "age": 10}, auth=[{"actor": userAccountName, "permission":"active"}])
-    assert(results[0])
-    apiNode.waitForTransactionInBlock(results[1]['transaction_id'])
+    def sendReadOnlyTrxOnThread(startId, numTrxs):
+        Print("start sendReadOnlyTrxOnThread")
 
-    # verify the return value (age) from read-only is the same as created.
-    Print("Send a read-only Get transaction to verify previous Insert")
-    results = sendTransaction('getage', {"user": userAccountName}, opts='--read')
-    assert(results[0])
-    assert(results[1]['processed']['action_traces'][0]['return_value_data'] == 10)
+        global errorInThread
+        errorInThread = False
+        try:
+           for i in range(numTrxs):
+               results = sendTransaction('age', {"user": userAccountName, "id": startId + i}, opts='--read')
+               assert(results[0])
+               assert(results[1]['processed']['action_traces'][0]['return_value_data'] == 25)
+        except Exception as e:
+            Print("Exception in sendReadOnlyTrxOnThread: ", e)
+            errorInThread = True
 
-    # verify non-read-only modification works
-    Print("Send a non-read-only Modify transaction")
-    results = sendTransaction('modify', {"user": userAccountName, "age": 25},        auth=[{"actor": userAccountName, "permission": "active"}])
-    assert(results[0])
-    apiNode.waitForTransactionInBlock(results[1]['transaction_id'])
+    def sendTrxsOnThread(startId, numTrxs, opts=None):
+        Print("sendTrxsOnThread: ", startId, numTrxs, opts)
 
-    # verify 'cleos push action getage "user": "user" --read' works
-    Print("Send a read-only Get action")
-    results = apiNode.pushMessage(testAccountName, 'getage', "{{\"user\": \"{}\"}}".format(userAccountName), opts='--read');
-    assert(results[0])
-    assert(results[1]['processed']['action_traces'][0]['return_value_data'] == 25)
+        global errorInThread
+        errorInThread = False
+        try:
+            for i in range(numTrxs):
+                results = sendTransaction('age', {"user": userAccountName, "id": startId + i}, auth=[{"actor": userAccountName, "permission":"active"}], opts=opts)
+                assert(results[0])
+        except Exception as e:
+            Print("Exception in sendTrxsOnThread: ", e)
+            errorInThread = True
+
+    def doRpc(resource, command, numRuns, fieldIn, expectedValue, code, payload={}):
+        global errorInThread
+        errorInThread = False
+        try:
+            for i in range(numRuns):
+                ret_json = apiNode.processUrllibRequest(resource, command, payload)
+                if code is None:
+                    assert(ret_json["payload"][fieldIn] is not None)
+                    if expectedValue is not None:
+                        assert(ret_json["payload"][fieldIn] == expectedValue)
+                else:
+                    assert(ret_json["code"] == code)
+        except Exception as e:
+            Print("Exception in doRpc: ", e)
+            errorInThread = True
+
+    def runReadOnlyTrxAndRpcInParallel(resource, command, fieldIn=None, expectedValue=None, code=None, payload={}):
+        Print("runReadOnlyTrxAndRpcInParallel: ", command)
+
+        numRuns = 10
+        trxThread = threading.Thread(target = sendReadOnlyTrxOnThread, args = (0, numRuns ))
+        rpcThread = threading.Thread(target = doRpc, args = (resource, command, numRuns, fieldIn, expectedValue, code, payload))
+        trxThread.start()
+        rpcThread.start()
+
+        trxThread.join()
+        rpcThread.join()
+        assert(not errorInThread)
+
+    def mixedOpsTest(opt=None):
+        Print("mixedOpsTest -- opt = ", opt)
+
+        numRuns = 300
+        readOnlyThread = threading.Thread(target = sendReadOnlyTrxOnThread, args = (0, numRuns ))
+        readOnlyThread.start()
+        sendTrxThread = threading.Thread(target = sendTrxsOnThread, args = (numRuns, numRuns, opt))
+        sendTrxThread.start()
+        pushBlockThread = threading.Thread(target = doRpc, args = ("chain", "push_block", numRuns, None, None, 202, {"block":"signed_block"}))
+        pushBlockThread.start()
+
+        readOnlyThread.join()
+        sendTrxThread.join()
+        pushBlockThread.join()
+        assert(not errorInThread)
+
+    def sendMulReadOnlyTrx(numThreads):
+        threadList = []
+        num_trxs_per_thread = 500
+        for i in range(numThreads):
+            thr = threading.Thread(target = sendReadOnlyTrxOnThread, args = (i * num_trxs_per_thread, num_trxs_per_thread ))
+            thr.start()
+            threadList.append(thr)
+        for thr in threadList:
+            thr.join()
+
+    def basicTests():
+        Print("Insert a user")
+        results = sendTransaction('insert', {"user": userAccountName, "id": 1, "age": 10}, auth=[{"actor": userAccountName, "permission":"active"}])
+        assert(results[0])
+        apiNode.waitForTransactionInBlock(results[1]['transaction_id'])
+
+        # verify the return value (age) from read-only is the same as created.
+        Print("Send a read-only Get transaction to verify previous Insert")
+        results = sendTransaction('getage', {"user": userAccountName}, opts='--read')
+        assert(results[0])
+        assert(results[1]['processed']['action_traces'][0]['return_value_data'] == 10)
+
+        # verify non-read-only modification works
+        Print("Send a non-read-only Modify transaction")
+        results = sendTransaction('modify', {"user": userAccountName, "age": 25}, auth=[{"actor": userAccountName, "permission": "active"}])
+        assert(results[0])
+        apiNode.waitForTransactionInBlock(results[1]['transaction_id'])
+
+    def multiReadOnlyTests():
+        Print("Verify multiple read-only Get actions work after Modify")
+        sendMulReadOnlyTrx(numThreads=5)
+
+    def chainApiTests():
+        # verify chain APIs can run in parallel with read-ony transactions
+        runReadOnlyTrxAndRpcInParallel("chain", "get_info", "server_version")
+        runReadOnlyTrxAndRpcInParallel("chain", "get_consensus_parameters", "chain_config")
+        runReadOnlyTrxAndRpcInParallel("chain", "get_activated_protocol_features", "activated_protocol_features")
+        runReadOnlyTrxAndRpcInParallel("chain", "get_block", "block_num", expectedValue=1, payload={"block_num_or_id":1})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_block_info", "block_num", expectedValue=1, payload={"block_num":1})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_account", "account_name", expectedValue=userAccountName, payload = {"account_name":userAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_code", "account_name", expectedValue=testAccountName, payload = {"account_name":testAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_code_hash", "account_name", expectedValue=testAccountName, payload = {"account_name":testAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_abi", "account_name", expectedValue=testAccountName, payload = {"account_name":testAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_raw_code_and_abi", "account_name", expectedValue=testAccountName, payload = {"account_name":testAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_raw_abi", "account_name", expectedValue=testAccountName, payload = {"account_name":testAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_producers", "rows", payload = {"json":"true","lower_bound":""})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_table_rows", code=500, payload = {"json":"true","table":"noauth"})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_table_by_scope", fieldIn="rows", payload = {"json":"true","table":"noauth"})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_currency_balance", code=200, payload = {"code":"eosio.token", "account":testAccountName})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_currency_stats", fieldIn="SYS", payload = {"code":"eosio.token", "symbol":"SYS"})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_required_keys", code=400)
+        runReadOnlyTrxAndRpcInParallel("chain", "get_transaction_id", code=200, payload = {"ref_block_num":"1"})
+        runReadOnlyTrxAndRpcInParallel("chain", "push_block", code=202, payload = {"block":"signed_block"})
+        runReadOnlyTrxAndRpcInParallel("chain", "get_producer_schedule", "active")
+        runReadOnlyTrxAndRpcInParallel("chain", "get_scheduled_transactions", "transactions", payload = {"json":"true","lower_bound":""})
+
+    def netApiTests():
+        # NET APIs
+        runReadOnlyTrxAndRpcInParallel("net", "status", code=201, payload = "localhost")
+        runReadOnlyTrxAndRpcInParallel("net", "connections", code=201)
+        runReadOnlyTrxAndRpcInParallel("net", "connect", code=201, payload = "localhost")
+        runReadOnlyTrxAndRpcInParallel("net", "disconnect", code=201, payload = "localhost")
+
+    basicTests()
+
+    if args.read_only_threads > 0: # Save test time. No need to run other tests if multi-threaded is not enabled
+        multiReadOnlyTests()
+        chainApiTests()
+        netApiTests()
+        mixedOpsTest()
 
     testSuccessful = True
 finally: