roadmapping

[serapath]

@todo

• fix the compiler for solidity version 0.5 so that play-editor works again
• temporarily we could just add some code here https://github.com/playproject-io/play-ed/blob/master/src/editor-solidity.js#L49 to filter out all new compiler versions 0.5+, because also remix is having issue
• write down a few example solidity contracts to use them for manual testing
  • contract with all possible "solidity features" (e.g. all data & function types, input, etc...)
• specify a proposal for a the solcjs module api
  • add a section in solc-js#README.md to show all available properties and methods
  • add comments and descriptions about all properties of API and what they are about
  • discuss standard with team and refine until everyone is satisfied
• refactor code
  • modularize compiler code
  • implement module API as specified below
  • minimize code
    1. solcjs.version2url ---> await solcjs.versions
    2. solcjs ---> await solcjs
    3. compiler ---> await compiler
    4. add compiler.version function return { name: '', url: ''}
       solc-js#9: use webworker
• specify proposal for a useful standard compiler output format
  • solc-js#3: design compiler standard output format
  • add a section in solc-js#README.md to show the output JSON
  • add comments and descriptions about all properties of the output format and what it is about
  • discuss standard with team and refine until everyone is satisfied
  • implement an adapter to make sure all compiler versions produce the correct output format
• solc-js#4: re-use original solc test suite
• solc-js#5: enable hooks for fetching code behind solidity import statements
• solc-js#6: add solidity import resolvers
• refactor smartcontract-ui to accept the new output format and works with all compiler versions
• solc-js#7: make solc-js an "(un)installable" command
• solc-js#8: add other solidity compilers as commands
• solc-js#2 (add vyper compiler)

---

review of refactoring

1. i like those:
   https://www.npmjs.com/package/ajax-caching
   https://www.npmjs.com/package/solc-version

2. not yet sure - need to see an example how they work with the compiler import callback
   https://www.npmjs.com/package/solc-import
   https://www.npmjs.com/package/solc-resolver
   (also how they would look like if they would use many small resolver modules for each backend (e.g. swarm, github, ipfs, etc...)

3. i wish this one would be much more raw and minimal:
   https://www.npmjs.com/package/solcjs-core

4. I am not yet sure about this one - depends on solcjs-core which would be cool if it was more minimal
   https://www.npmjs.com/package/solcjs-lightweight

implement a small resolver module

const ResolverEngine = require('solc-resolver').ResolverEngine;
let resolverEngine = new ResolverEngine();
let resolveGithub = require('resolve-github');
resolverEngine.addResolver(resolveGithub);
let resolveIpfs = require('resolve-ipfs');
resolverEngine.addResolver(resolveIpfs);

const getImportContent = async function (path) {
  return myDB.has(path) ? myDB.get(path) : await resolverEngine.require(path);
};

let output = await compiler(sourceCode, getImportContent);

The goal is:

1. to develop a nice "user interface" for developers who want to use a solidity compiler version, which means a standard output format and convert output of all solc-bin versions into this format
2. document the cross-compiler-version standard format in the modules README

The solidity compiler is developed in C++. There are autogenerated transpilations into Javascript in https://github.com/ethereum/solc-bin. Every time a new solidity compiler version is released, the soc-bin repository is updated. We maintain solc-js as a wrapper module around the autogenerated solidity compiler versions.

Updating the ABI

The ABI generated by Solidity versions can differ slightly, due to new features introduced. There is a tool included which aims to translate the ABI generated by an older Solidity version to conform to the latest standard.

It can be used as:

var abi = require('solc/abi')

var inputABI = [{"constant":false,"inputs":[],"name":"hello","outputs":[{"name":"","type":"string"}],"payable":false,"type":"function"}]
var outputABI = abi.update('0.3.6', inputABI)
// Output contains: [{"constant":false,"inputs":[],"name":"hello","outputs":[{"name":"","type":"string"}],"payable":true,"type":"function"},{"type":"fallback","payable":true}]

Formatting old JSON assembly output

There is a helper available to format old JSON assembly output into a text familiar to earlier users of Remix IDE.

var translate = require('solc/translate')

// assemblyJSON refers to the JSON of the given assembly and sourceCode is the source of which the assembly was generated from
var output = translate.prettyPrintLegacyAssemblyJSON(assemblyJSON, sourceCode)

Linking Bytecode

When using libraries, the resulting bytecode will contain placeholders for the real addresses of the referenced libraries. These have to be updated, via a process called linking, before deploying the contract.

The linker module (require('solc/linker')) offers helpers to accomplish this.

The linkBytecode method provides a simple helper for linking:

var linker = require('solc/linker')

bytecode = linker.linkBytecode(bytecode, { 'MyLibrary': '0x123456...' })

(Note: linkBytecode is also exposed via solc as solc.linkBytecode, but this usage is deprecated.)

As of Solidity 0.4.11 the compiler supports standard JSON input and output which outputs a link references map. This gives a map of library names to offsets in the bytecode to replace the addresses at. It also doesn't have the limitation on library file and contract name lengths.

There is a method available in the linker module called findLinkReferences which can find such link references in bytecode produced by an older compiler:

var linker = require('solc/linker')

var linkReferences = linker.findLinkReferences(bytecode)

Using the Latest Development Snapshot

By default, the npm version is only created for releases. This prevents people from deploying contracts with non-release versions because they are less stable and harder to verify. If you would like to use the latest development snapshot (at your own risk!), you may use the following example code.

var solc = require('solc')

// getting the development snapshot
solc.loadRemoteVersion('latest', function (err, solcSnapshot) {
	if (err) {
		// An error was encountered, display and quit
	}
	var output = solcSnapshot.compile("contract t { function g() {} }", 1)
})

Using a Legacy Version

In order to compile contracts using a specific version of Solidity, the solc.loadRemoteVersion(version, callback) method is available. This returns a new solc object that uses a version of the compiler specified.

You can also load the "binary" manually and use setupMethods to create the familiar wrapper functions described above:
var solc = solc.setupMethods(require("/my/local/soljson.js")).

---

From version 0.4.20

Starting from version 0.4.20 a Semver compatible version number can be retrieved on every compiler release, including old ones, using the semver() method.

From version 0.4.11

Starting from version 0.4.11 there is a new entry point named compileStandardWrapper() which supports Solidity's standard JSON input and output. It also maps old compiler output to it.

var solc = require('solc')

// 'input' is a JSON string corresponding to the "standard JSON input" as described in the link above
// 'findImports' works as described above
var output = solc.compileStandardWrapper(input, findImports)
// Ouput is a JSON string corresponding to the "standard JSON output"

There is also a direct method, compileStandard, which is only present on recent compilers and works the same way. compileStandardWrapper is preferred however because it provides the same interface for old compilers.

From version 0.2.1

Starting from version 0.2.1, a callback is supported to resolve missing imports as follows:

var solc = require('solc')
var input = {
	'cont.sol': 'import "lib.sol"; contract x { function g() { L.f(); } }'
}
function findImports (path) {
	if (path === 'lib.sol')
		return { contents: 'library L { function f() returns (uint) { return 7; } }' }
	else
		return { error: 'File not found' }
}
var output = solc.compile({ sources: input }, 1, findImports)
for (var contractName in output.contracts)
	console.log(contractName + ': ' + output.contracts[contractName].bytecode)

The compile() method always returns an object, which can contain errors, sources and contracts fields. errors is a list of error mesages.

From version 0.1.6

Starting from version 0.1.6, multiple files are supported with automatic import resolution by the compiler as follows:

var solc = require('solc')
var input = {
	'lib.sol': 'library L { function f() returns (uint) { return 7; } }',
	'cont.sol': 'import "lib.sol"; contract x { function g() { L.f(); } }'
}
var output = solc.compile({ sources: input }, 1)
for (var contractName in output.contracts)
	console.log(contractName + ': ' + output.contracts[contractName].bytecode)

Note that all input files that are imported have to be supplied, the compiler will not load any additional files on its own.

From early versions

It can also be included and used in other projects:

var solc = require('solc')
var input = 'contract x { function g() {} }'
// Setting 1 as second paramateractivates the optimiser
var output = solc.compile(input, 1)
for (var contractName in output.contracts) {
	// code and ABI that are needed by web3
	console.log(contractName + ': ' + output.contracts[contractName].bytecode)
	console.log(contractName + '; ' + JSON.parse(output.contracts[contractName].interface))
}

MORE

// /******************************************************************************
//   VERSIONS
// ******************************************************************************/
//
// function v0_0_0_plus () {
//   // From early versions
//   // it can also be included and used in other projects
//   var solc = require('solc')
//   var input = 'contract x { function g() {} }'
//   // Setting 1 as second paramateractivates the optimiser
//   var output = solc.compile(input, 1)
//   for (var contractName in output.contracts) {
//       // code and ABI that are needed by web3
//       console.log(contractName + ': ' + output.contracts[contractName].bytecode)
//       console.log(contractName + '; ' + JSON.parse(output.contracts[contractName].interface))
//   }
// }
// function v0_1_6_plus () {
//   // From version 0.1.6
//   // multiple files are supported with automatic import resolution
//   // by the compiler as follows
//   var input = {
//       'lib.sol': 'library L { function f() returns (uint) { return 7; } }',
//       'cont.sol': 'import "lib.sol"; contract x { function g() { L.f(); } }'
//   }
//   var output = solc.compile({ sources: input }, 1)
//   for (var contractName in output.contracts)
//       console.log(contractName + ': ' + output.contracts[contractName].bytecode)
// }
// function v0_2_1_plus () {
//   // From version 0.2.1
//   // a callback is supported to resolve missing imports as follows
//   /*
//   The compile() method always returns an object, which can contain errors,
//   sources and contracts fields.
//   errors is a list of error mesages
//   */
//   var solc = require('solc')
//   var input = {
//       'cont.sol': 'import "lib.sol"; contract x { function g() { L.f(); } }'
//   }
//   function findImports (path) {
//       if (path === 'lib.sol')
//           return { contents: 'library L { function f() returns (uint) { return 7; } }' }
//       else
//           return { error: 'File not found' }
//   }
//   var output = solc.compile({ sources: input }, 1, findImports)
//   for (var contractName in output.contracts)
//       console.log(contractName + ': ' + output.contracts[contractName].bytecode)
// }
// function v0_4_11_plus () {
//   // From version 0.4.11
//   // Starting from version 0.4.11 there is a new entry point
//   // named compileStandardWrapper() which supports
//   // https://solidity.readthedocs.io/en/develop/using-the-compiler.html#compiler-input-and-output-json-description
//   // => Solidity's standard JSON input and output.
//   // It also maps old compiler output to it.
//   // => outputs a link references map
//   // => This gives a map of library names to offsets in the bytecode to replace the addresses at
//   //  It also doesn't have the limitation on library file and contract name lengths
//   /*
//   There is also a direct method, compileStandard, which is only present on
//   recent compilers and works the same way. compileStandardWrapper
//   is preferred however because it provides the same interface for old compilers.
//   */
//   var solc = require('solc')
//
//   // 'input' is a JSON string corresponding to the "standard JSON input" as described in the link above
//   // 'findImports' works as described above
//   var output = solc.compileStandardWrapper(input, findImports)
//   // Ouput is a JSON string corresponding to the "standard JSON output"
// }
// function v0_4_20_plus () {
//   // From version 0.4.20
//   /*
//   Starting from version 0.4.20 a Semver compatible version number can be
//   retrieved on every compiler release, including old ones,
//   using the semver() method.
//   */
// }
//
// /******************************************************************************
//   LOAD COMPILER
// ******************************************************************************/
// function loadCompiler (version, next) {
//   // API:
//   // In order to compile contracts using a specific version of Solidity,
//   // the solc.loadRemoteVersion(version, callback) method is available.
//   // This returns a new solc object that uses a version of the compiler specified
//   //
//   // You can also load the "binary" manually and use setupMethods
//   // to create the familiar wrapper functions described above:
//   // var solc = solc.setupMethods(require("/my/local/soljson.js"))
//
//   var solc = require('solc')
//   // getting the development snapshot
//   solc.loadRemoteVersion('latest', function (err, solcSnapshot) {
//       if (err) {
//           // An error was encountered, display and quit
//       }
//       var output = solcSnapshot.compile("contract t { function g() {} }", 1)
//       next(null, solcSnapshot)
//   })
// }
// /******************************************************************************
//   LINKING BYTECODE
// ******************************************************************************/
// function linkByteCode ({ unlinkedByteCode, libAddressMap }, next) {
//   // LINKING BYTECODE
//   // When using libraries, the resulting bytecode will contain placeholders
//   // for the real addresses of the referenced libraries.
//   // These have to be updated, via a process called linking,
//   // before deploying the contract.
//
//   // The linker module
//   // (require('solc/linker'))
//   // offers helpers to accomplish this.
//
//   // The linkBytecode method provides a simple helper for linking:
//   var linker = require('solc/linker')
//   libAddressMap = libAddressMap || { 'MyLibrary': '0x123456...' }
//   bytecode = linker.linkBytecode(unlinkedByteCode, libAddressMap)
//   // Note: linkBytecode is also exposed via solc as solc.linkBytecode,
//   // but this usage is deprecated.
//
//   /*
//     There is a method available in the linker module called findLinkReferences
//     which can find such link references in bytecode produced
//     by an older compiler:
//   */
//   var linker = require('solc/linker')
//   var linkReferences = linker.findLinkReferences(bytecode)
//
//   const linkedByteCode = bytecode
//   next(null, linkedByteCode)
// }
// /******************************************************************************
//   FORMAT ASSEMBLY JSON
// ******************************************************************************/
// function formatJSON ({ assemblyJSON, sourceCode }, next) {
//   // Formatting old JSON assembly output
//   //
//   // There is a helper available to format old JSON assembly output
//   // into a text familiar to earlier users of Remix IDE
//   var translate = require('solc/translate')
//
//   // assemblyJSON refers to the JSON of the given assembly and sourceCode is the source of which the assembly was generated from
//   var output = translate.prettyPrintLegacyAssemblyJSON(assemblyJSON, sourceCode)
//
//   next(null, output)
// }
// /******************************************************************************
//   UPDATE ABI
// ******************************************************************************/
// function updateABI ({ version, oldABI }, next) {
//   // UPDATING THE ABI
//   /*
//     The ABI generated by Solidity versions can differ slightly,
//     due to new features introduced.
//     There is a tool included which aims to translate the ABI generated
//     by an older Solidity version to conform to the latest standard
//   */
//   var abi = require('solc/abi')
//
//   var inputABI = [{"constant":false,"inputs":[],"name":"hello","outputs":[{"name":"","type":"string"}],"payable":false,"type":"function"}]
//   var outputABI = abi.update('0.3.6', inputABI)
//   // Output contains: [{"constant":false,"inputs":[],"name":"hello","outputs":[{"name":"","type":"string"}],"payable":true,"type":"function"},{"type":"fallback","payable":true}]
//
//   const newABI = outputABI
//   next(null, newABI)
// }

---

/src/solc-js.js
/src/lib/versions/index.js
/src/lib/versions/package.json
/src/lib/versions/versions2url.js
/src/lib/versions/versions.js
/src/lib/cache/index.js



/src/lib/ajax-cache/package.json
/src/lib/ajax-cache/cache.js
/src/lib/ajax-cache/node_modules/ajax/index.js


-----------

/src/node_modules/cache-ajax
/src/node_modules/cache-ajax/index.js
/src/node_modules/cache-ajax/test/test.js
/src/node_modules/cache-ajax/package.json
/src/node_modules/cache-ajax/

/solcjs/$>cd src/node_modules/cache-ajax/


// 100 LOC

/solcjs/src/node_modules/cache-ajax/$> npm run test

// MAIN FILE
// ./src/solc-js.js
const cache = require('ajax-cache') // ./src/node_modules/ajax-cache/**/*

console.log(cache)

Also:
These are all compiler tests the solidity developer team updates regularly and we should see that our compiler can pass them all: https://github.com/ethereum/solc-js/tree/master/test

---

write down a few example solidity contracts to use them for manual testing

pragma solidity ^0.5.3;

contract Sample {
  bool bool1 = true;
  bool bool2 = false;
  
  int num = 0;
  int8 num8 = 8;
  int16 num16 = 16;
  int32 num32 = 32;
  int40 num40 = 40;
  int48 num48 = 48;
  int56 num56 = 56;
  int64 num64 = 64;
  int72 num72 = 72;
  int128 num128 = 128;
  int256 num256 = 256;
  
  uint unum = 0;
  uint8 unum8 = 255;
  uint16 unum16 = 16;
  uint32 unum32 = 32;
  uint256 unum256 = 256;
  
  address addr;
  
  bytes myBytes;
  bytes1 myBytes1;
  bytes2 myBytes2;
  bytes3 myBytes3;
  bytes8 myBytes8;
  bytes32 myBytes32;
  
  string name;
  
  enum State { Start, Pending, End }
  State state = State.Start;
  
  uint[] myArray = new uint[](5);
  int[5] myArray2;
  address[] funderIndexs;
  
  struct Person {
    string name;
    uint height;
  }
  
  Person customer;
  
  mapping (address => Funder) funders;

  struct Funder {
    address addr;
    uint amount;
    uint createdAt;
    string name;
  }
  
  function function1() public {}
  
  function setBoolean(bool _a) public returns(bool) {
      bool1 = _a;
      return bool1;
  }
  
  function getNumber() public returns(uint) {
      return unum;
  }
  
  function setNumber(uint _a) public returns(uint) {
      unum = _a;
      return unum;
  }
  
  function setOwner(address _addr) public returns(address) {
      addr = _addr;
      return addr;
  }
  
  function setBytes1(bytes1 _a) public returns(bytes1) {
      myBytes1 = _a;
      return myBytes1;
  }
  
  function setBytes(bytes memory _a) public returns(bytes memory) {
      myBytes = _a;
      return myBytes;
  }
  
  function setString(string memory _a) public returns(string memory) {
      name = _a;
      return name;
  }
  
  function getState() view public returns (uint) {
    return uint(state);
  }
  
  function setPerson(string memory _name, uint height) public {
    customer = Person({ name: 'alincode', height: 160 });
  }
      
  function fund(string memory _name) public payable {
    funders[msg.sender] = Funder(msg.sender, msg.value, now, _name);
  }
  
  function addMyArray(uint _a) public {
      myArray.push(_a);
  }
  
}