Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial benchmarks for intersection types + a bit of speedup #11924

Merged
merged 29 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
086d024
Ensure isAllTypes is compilation constant
JaroslavTulach Dec 19, 2024
a33965d
Using internal MultiType to represent Type[] but with guaranteed ==
JaroslavTulach Dec 19, 2024
171c799
Merge remote-tracking branch 'origin/develop' into wip/jtulach/MultiT…
JaroslavTulach Dec 19, 2024
775595c
Prefer partialEvaluationConstant assert
JaroslavTulach Dec 19, 2024
e2760ff
Benchmarks for intersection types
JaroslavTulach Dec 20, 2024
630ec62
Usage of Map & co. must be behind @TruffleBoundary
JaroslavTulach Dec 20, 2024
8defcfe
Unify findInteropTypeValue
JaroslavTulach Dec 20, 2024
d44d30e
Inline cache to find index of a type
JaroslavTulach Dec 20, 2024
7b6d364
Use EnsoMultiValue.NewNode to allocate new instances of EnsoMultiValue
JaroslavTulach Dec 20, 2024
bda398a
Basic specializations for NewNode
JaroslavTulach Dec 20, 2024
56b24aa
Splitting the FindIndexNode and caching requests for newNode
JaroslavTulach Dec 20, 2024
2dcc2c4
Just ask only for types the value 'has been cast to'
JaroslavTulach Dec 20, 2024
ee080a3
Provide cachedTypes as the first argument to activate the caches
JaroslavTulach Dec 20, 2024
53c2222
AllOfTypesCheckNode needs cached EnsoMultiValue.NewNode to allocate E…
JaroslavTulach Dec 20, 2024
9567257
Moving EnsoMultiType into outer scope
JaroslavTulach Dec 20, 2024
6bfdbf9
Sum re field of a Complex object in a Vector is the base benchmark
JaroslavTulach Dec 21, 2024
4dacf53
Cache dispatch on EnsoMultiValue.getDispatchId
JaroslavTulach Dec 21, 2024
ebe0553
Turing allTypesWith method into EnsoMultiType.AllTypesWith node
JaroslavTulach Dec 28, 2024
2301f9b
Only assert valid payload
JaroslavTulach Dec 30, 2024
8d5452c
Speeding up non-reordering reorderOnly case twice
JaroslavTulach Dec 30, 2024
ed8799c
Merging with develop and resolving conflicts
JaroslavTulach Dec 30, 2024
615b600
Don't use keyword as variable name
JaroslavTulach Dec 30, 2024
a68db22
Assert there is no intersection between dispatch and extra types
JaroslavTulach Dec 30, 2024
98ebc39
Merge remote-tracking branch 'origin/develop' into wip/jtulach/MultiT…
JaroslavTulach Jan 4, 2025
a9f57f0
Avoiding duplications when Number & Integer & Float and co.
JaroslavTulach Jan 4, 2025
035b2be
Merge branch 'develop' into wip/jtulach/MultiType11846
mergify[bot] Jan 4, 2025
3f0a3e3
Merge branch 'develop' into wip/jtulach/MultiType11846
mergify[bot] Jan 6, 2025
9503f79
Merge branch 'develop' into wip/jtulach/MultiType11846
mergify[bot] Jan 7, 2025
f515d8a
Merge branch 'develop' into wip/jtulach/MultiType11846
mergify[bot] Jan 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
package org.enso.interpreter.bench.benchmarks.semantic;

import java.util.concurrent.TimeUnit;
import org.enso.common.MethodNames;
import org.enso.compiler.benchmarks.Utils;
import org.graalvm.polyglot.Value;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.BenchmarkParams;
import org.openjdk.jmh.infra.Blackhole;

/**
* These benchmarks compare performance of {@link EnsoMultiValue}. They create a vector in a certain
* configuration representing numbers and then they perform {@code sum} operation on it.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Akirathan, @hubertp, @4e6 (and also @radeusgd):

  • these are the benchmarks for intersection types
  • a Vector is created by make_vector typ method
  • various types include Integer, Float, Complex as non-intersection type benchmarks to compare to
  • then there are various types mixing Float and Complex together into different intersection types

Ideally the intersection types benchmark results should be close to base benchmark sumOverComplexBaseBenchmark0 with some "reasonable overhead".

Are you OK with these benchmarks?

*/
@BenchmarkMode(Mode.AverageTime)
@Fork(1)
@Warmup(iterations = 3, time = 5, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class MultiValueBenchmarks {
private Value arrayOfNumbers;
private Value sum;
private Value self;
private final long length = 100000;

@Setup
public void initializeBenchmark(BenchmarkParams params) throws Exception {
var ctx = Utils.createDefaultContextBuilder().build();
var code =
"""
from Standard.Base import Vector, Float, Number, Integer

type Complex
private Number re:Float im:Float

Complex.from (that:Number) = Complex.Number that 0

sum arr =
go acc i = if i >= arr.length then acc else
v = arr.at i
sum = acc + v
@Tail_Call go sum i+1
go 0 0

sum_re arr =
go acc i = if i >= arr.length then acc else
v = arr.at i . re
sum = acc + v
@Tail_Call go sum i+1
go 0 0

make_vector type n =
Copy link
Member Author

@JaroslavTulach JaroslavTulach Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dedicated benchmark for EnsoMultiValue instances based on the idea of ArrayProxy one. Currently the more complicated benchmarks refuse to compile and the compiler bails out. The initial results are:

sbt:enso> runtime-benchmarks/benchOnly MultiValueBenchmarks
Benchmark                                                 Mode  Cnt    Score    Error  Units
MultiValueBenchmarks.sumOverComplexAndFloat5              avgt    5  214.330 ±  4.179  ms/op
MultiValueBenchmarks.sumOverComplexFloatRecastedToFloat3  avgt    5  219.803 ± 11.872  ms/op
MultiValueBenchmarks.sumOverFloat1                        avgt    5    0.079 ±  0.006  ms/op
MultiValueBenchmarks.sumOverFloatAndComplex6              avgt    5  219.525 ±  6.393  ms/op
MultiValueBenchmarks.sumOverFloatComplexRecastedToFloat4  avgt    5  203.788 ±  9.843  ms/op
MultiValueBenchmarks.sumOverInteger0                      avgt    5    0.074 ±  0.001  ms/op

After 630ec62 the results are better:

MultiValueBenchmarks.sumOverComplexAndFloat5              avgt    5  30.109 ± 0.661  ms/op
MultiValueBenchmarks.sumOverComplexFloatRecastedToFloat3  avgt    5  26.988 ± 0.446  ms/op
MultiValueBenchmarks.sumOverFloat1                        avgt    5   0.078 ± 0.003  ms/op
MultiValueBenchmarks.sumOverFloatAndComplex6              avgt    5  27.821 ± 0.856  ms/op
MultiValueBenchmarks.sumOverFloatComplexRecastedToFloat4  avgt    5  27.961 ± 0.263  ms/op
MultiValueBenchmarks.sumOverInteger0                      avgt    5   0.078 ± 0.002  ms/op

and there are no bailouts. Time to really speed things up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vector.new n i->
r = 3 + 5*i
case type of
0 -> r:Complex
1 -> r:Integer
2 -> r:Float
3 ->
c = r:Complex&Float
c:Float
4 ->
c = r:Float&Complex
c:Float
5 -> r:Complex&Float
6 -> r:Float&Complex
""";
var benchmarkName = SrcUtil.findName(params);
var src = SrcUtil.source(benchmarkName, code);
int type = Integer.parseInt(benchmarkName.substring(benchmarkName.length() - 1));

var module = ctx.eval(src);
this.self = module.invokeMember(MethodNames.Module.GET_ASSOCIATED_TYPE);
var makeVector = module.invokeMember("get_method", self, "make_vector");
this.arrayOfNumbers = makeVector.execute(self, type, length);
this.sum =
module.invokeMember(MethodNames.Module.EVAL_EXPRESSION, type == 0 ? "sum_re" : "sum");
}

/**
* The <b>base benchmark</b> for this suite. Measures how much it takes to access an Atom in a
* Vector, read {@code re:Float} field out of it and sum all of them together.
*/
@Benchmark
Copy link
Member Author

@JaroslavTulach JaroslavTulach Dec 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now the base benchmark. The results after 4dacf53 are:

# the base one
MultiValueBenchmarks.sumOverComplexBaseBenchmark0         avgt    5  0.139 ± 0.013  ms/op

# these two are supposed to be faster
MultiValueBenchmarks.sumOverInteger1                      avgt    5  0.065 ± 0.003  ms/op
MultiValueBenchmarks.sumOverFloat2                        avgt    5  0.073 ± 0.002  ms/op

# these should catch up with sumOverComplexBaseBenchmark0 one day
MultiValueBenchmarks.sumOverComplexAndFloat5              avgt    5  8.580 ± 0.326  ms/op
MultiValueBenchmarks.sumOverComplexFloatRecastedToFloat3  avgt    5  9.118 ± 0.483  ms/op
MultiValueBenchmarks.sumOverFloatAndComplex6              avgt    5  8.110 ± 0.160  ms/op
MultiValueBenchmarks.sumOverFloatComplexRecastedToFloat4  avgt    5  9.393 ± 0.648  ms/op

still 60 times slower than it should be.

Copy link
Member Author

@JaroslavTulach JaroslavTulach Dec 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The biggest slowdown is currently in reorderOnly branch:

reorderOnly

It seems to always allocate new arrays. Rather it should treat EnsoMultiType as compilation constants have everything ready from previous run.

Attempting to make things better in a follow up PR:

public void sumOverComplexBaseBenchmark0(Blackhole matter) {
performBenchmark(matter);
}

/**
* Working with {@code Integer} should be the fastest. The plus operation on integer are faster
* than those on {@code Float} and moreover the {@code Vector} has a special representation when
* full of {@code long} values.
*/
@Benchmark
public void sumOverInteger1(Blackhole matter) {
performBenchmark(matter);
}

/**
* Working with {@code Float} should be also fast. The {@code Vector} has a special representation
* when full of {@code double} values that increases cache locality.
*/
@Benchmark
public void sumOverFloat2(Blackhole matter) {
performBenchmark(matter);
}

//
// Following benchmarks shall catch up with the base benchmark
//

@Benchmark
public void sumOverComplexFloatRecastedToFloat3(Blackhole matter) {
performBenchmark(matter);
}

@Benchmark
public void sumOverFloatComplexRecastedToFloat4(Blackhole matter) {
performBenchmark(matter);
}

@Benchmark
public void sumOverComplexAndFloat5(Blackhole matter) {
performBenchmark(matter);
}

@Benchmark
public void sumOverFloatAndComplex6(Blackhole matter) {
performBenchmark(matter);
}

private void performBenchmark(Blackhole matter) throws AssertionError {
var resultValue = sum.execute(arrayOfNumbers);
if (!resultValue.fitsInLong()) {
throw new AssertionError("Shall be a long: " + resultValue);
}
long result = resultValue.asLong();
long expectedResult = length * 3L + (5L * (length * (length - 1L) / 2L));
boolean isResultCorrect = result == expectedResult;
if (!isResultCorrect) {
throw new AssertionError("Expecting " + expectedResult + " but was " + result);
}
matter.consume(result);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,19 @@ private static void registerValue(
var rawValue = ContextUtils.unwrapValue(ctx(), polyValue);
var rawType = ContextUtils.unwrapValue(ctx(), t);
if (rawType instanceof Type type) {
var singleMultiValue = EnsoMultiValue.create(new Type[] {type}, 1, new Object[] {rawValue});
var singleMultiValue =
EnsoMultiValue.NewNode.getUncached()
.newValue(new Type[] {type}, 1, new Object[] {rawValue});
var n = t.getMetaSimpleName();
data.add(new Object[] {singleMultiValue, n, 0});
var rawInt = (Type) ContextUtils.unwrapValue(ctx(), g.typeInteger());
var secondMultiValue =
EnsoMultiValue.create(new Type[] {rawInt, type}, 2, new Object[] {5L, rawValue});
EnsoMultiValue.NewNode.getUncached()
.newValue(new Type[] {rawInt, type}, 2, new Object[] {5L, rawValue});
data.add(new Object[] {secondMultiValue, n, 1});
var firstMultiValue =
EnsoMultiValue.create(new Type[] {type, rawInt}, 2, new Object[] {rawValue, 6L});
EnsoMultiValue.NewNode.getUncached()
.newValue(new Type[] {type, rawInt}, 2, new Object[] {rawValue, 6L});
data.add(new Object[] {firstMultiValue, n, 0});
} else {
if (!t.isHostObject()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,9 @@ public void multiValueWithHiddenType() {
new Type[] {
ensoCtx.getBuiltins().number().getInteger(), ensoCtx.getBuiltins().text()
};
var multi = EnsoMultiValue.create(types, 1, new Object[] {42L, "Meaning"});
var multi =
EnsoMultiValue.NewNode.getUncached()
.newValue(types, 1, new Object[] {42L, "Meaning"});
var arr = (Object[]) testTypesCall.call(multi, true);
var allTypes = (Type[]) arr[1];
assertEquals("Two types", 2, allTypes.length);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,11 @@ public void avoidDoubleWrappingOfEnsoMultiValue() {
() -> {
var builtins = ContextUtils.leakContext(ctx).getBuiltins();
var m1 =
EnsoMultiValue.create(
new Type[] {builtins.text(), builtins.number().getInteger()},
2,
new Object[] {"Hi", 42});
EnsoMultiValue.NewNode.getUncached()
.newValue(
new Type[] {builtins.text(), builtins.number().getInteger()},
2,
new Object[] {"Hi", 42});
assertEquals("Text & Integer", m1.toDisplayString(true));

var res = convert.call(m1);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,9 @@ private static void registerValue(
if (rawT1 instanceof Type typ1 && rawT2 instanceof Type typ2) {
var r1 = ContextUtils.unwrapValue(ctx, v1);
var r2 = ContextUtils.unwrapValue(ctx, v2);
var both = EnsoMultiValue.create(new Type[] {typ1, typ2}, 2, new Object[] {r1, r2});
var both =
EnsoMultiValue.NewNode.getUncached()
.newValue(new Type[] {typ1, typ2}, 2, new Object[] {r1, r2});
data.add(new Object[] {both});
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,9 @@ public void testEqualityIntegerAndMultiValue() {
var intType = builtins.number().getInteger();
var textText = builtins.text();
var fourExtraText =
EnsoMultiValue.create(
new Type[] {intType, textText}, 1, new Object[] {4L, Text.create("Hi")});
EnsoMultiValue.NewNode.getUncached()
.newValue(
new Type[] {intType, textText}, 1, new Object[] {4L, Text.create("Hi")});

assertTrue("4 == 4t", equalityCheck(4L, fourExtraText));
assertFalse("5 != 4t", equalityCheck(5L, fourExtraText));
Expand Down Expand Up @@ -85,7 +86,7 @@ public void testEqualityTextAndExtraIntegerMultiValue() {
// x = _ : (Text & Integer) : Text
// e.g. multi value with Text and Integer, casted to Text only
//
var multiV = EnsoMultiValue.create(bothTypes, 1, text, integer);
var multiV = EnsoMultiValue.NewNode.getUncached().newValue(bothTypes, 1, text, integer);

assertTrue("'Hi' == multiV", equalityCheck(text, multiV));
assertFalse("'Ahoj' != multiV", equalityCheck(ahoj, multiV));
Expand All @@ -112,7 +113,8 @@ public void testEqualityIntegerAndMultiValueWithBoth() {
var textText = builtins.text();
var hi = Text.create("Hi");
var fourExtraText =
EnsoMultiValue.create(new Type[] {textText, intType}, 2, new Object[] {hi, 4L});
EnsoMultiValue.NewNode.getUncached()
.newValue(new Type[] {textText, intType}, 2, new Object[] {hi, 4L});

assertTrue("4 == 4t", equalityCheck(4L, fourExtraText));
assertFalse("5 != 4t", equalityCheck(5L, fourExtraText));
Expand All @@ -134,8 +136,9 @@ public void testEqualityIntegerAndMultiValueWithIntText() {
var intType = builtins.number().getInteger();
var textText = builtins.text();
var fourExtraText =
EnsoMultiValue.create(
new Type[] {intType, textText}, 2, new Object[] {4L, Text.create("Hi")});
EnsoMultiValue.NewNode.getUncached()
.newValue(
new Type[] {intType, textText}, 2, new Object[] {4L, Text.create("Hi")});

assertTrue("4 == 4t", equalityCheck(4L, fourExtraText));
assertFalse("5 != 4t", equalityCheck(5L, fourExtraText));
Expand All @@ -155,14 +158,17 @@ public void twoMultiValues() {
var intType = builtins.number().getInteger();
var textText = builtins.text();
var fourExtraText =
EnsoMultiValue.create(
new Type[] {intType, textText}, 1, new Object[] {4L, Text.create("Hi")});
EnsoMultiValue.NewNode.getUncached()
.newValue(
new Type[] {intType, textText}, 1, new Object[] {4L, Text.create("Hi")});
var fourExtraText2 =
EnsoMultiValue.create(
new Type[] {intType, textText}, 1, new Object[] {4L, Text.create("Hi")});
EnsoMultiValue.NewNode.getUncached()
.newValue(
new Type[] {intType, textText}, 1, new Object[] {4L, Text.create("Hi")});
var fiveExtraText =
EnsoMultiValue.create(
new Type[] {intType, textText}, 1, new Object[] {5L, Text.create("Hi")});
EnsoMultiValue.NewNode.getUncached()
.newValue(
new Type[] {intType, textText}, 1, new Object[] {5L, Text.create("Hi")});

assertFalse("!= for sure #1", equalityCheck(fiveExtraText, fourExtraText));
assertFalse("!= for sure #2", equalityCheck(fourExtraText, fiveExtraText));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@ abstract Object executeCheckOrConversion(
@ExplodeLoop
final boolean isAllTypes() {
Node p = this;
CompilerAsserts.compilationConstant(p);
CompilerAsserts.partialEvaluationConstant(p);
for (; ; ) {
if (p instanceof TypeCheckValueNode vn) {
CompilerAsserts.compilationConstant(vn);
CompilerAsserts.partialEvaluationConstant(vn);
var allTypes = vn.isAllTypes();
CompilerAsserts.compilationConstant(allTypes);
CompilerAsserts.partialEvaluationConstant(allTypes);
return allTypes;
}
p = p.getParent();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ final class AllOfTypesCheckNode extends AbstractTypeCheckNode {

@Children private AbstractTypeCheckNode[] checks;
@Child private TypesLibrary types;
@Child private EnsoMultiValue.NewNode newNode;

AllOfTypesCheckNode(String name, AbstractTypeCheckNode[] checks) {
super(name);
this.checks = checks;
this.types = TypesLibrary.getFactory().createDispatched(checks.length);
this.newNode = EnsoMultiValue.NewNode.create();
}

AbstractTypeCheckNode[] getChecks() {
Expand Down Expand Up @@ -52,7 +54,7 @@ Object executeCheckOrConversion(VirtualFrame frame, Object value, ExpressionNode
values[at] = result;
at++;
}
return EnsoMultiValue.create(valueTypes, valueTypes.length, values);
return newNode.newValue(valueTypes, valueTypes.length, values);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,23 +170,14 @@ ApplicationNode findConversionNode(ExpressionNode valueNode, Type[] allTypes) {
return null;
}

Type[] findType(TypeOfNode typeOfNode, Object v) {
final Type[] findType(TypeOfNode typeOfNode, Object v) {
return findType(typeOfNode, v, null);
}

Type[] findType(TypeOfNode typeOfNode, Object v, Type[] previous) {
final Type[] findType(TypeOfNode typeOfNode, Object v, Type[] previous) {;
if (v instanceof EnsoMultiValue multi) {
var all = typeOfNode.findAllTypesOrNull(multi, isAllTypes());
assert all != null;
var to = 0;
// only consider methodDispatchTypes and not "all types" of the multi value
for (var i = 0; i < all.length; i++) {
var typeOrNull = EnsoMultiValue.CastToNode.getUncached().findTypeOrNull(all[i], multi, false, false);
if (typeOrNull != null) {
all[to++] = all[i];
}
}
return Arrays.copyOf(all, to);
var all = typeOfNode.findAllTypesOrNull(multi, false);
return all;
}
if (v instanceof UnresolvedConstructor) {
return null;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ public ModuleScope getScope() {
* @param type the type for which this symbol should be resolved
* @return the resolved function definition and type it was resolved in, or null if not found
*/
@TruffleBoundary
public Pair<Function, Type> resolveFor(Node node, Type type) {
if (type != null) {
for (var current : type.allTypes(EnsoContext.get(node))) {
Expand Down
Loading
Loading