Skip to content

Commit

Permalink
Update Arc-Lang to version 1
Browse files Browse the repository at this point in the history
  • Loading branch information
segeljakt committed Aug 16, 2023
1 parent 8d049bd commit aa7f604
Show file tree
Hide file tree
Showing 532 changed files with 121,706 additions and 16,765 deletions.
26 changes: 26 additions & 0 deletions .github/containers/ubuntu/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM ubuntu:18.04

# Add repos
RUN sudo apt update
RUN sudo apt install software-properties-common
RUN add-apt-repository ppa:avsm/ppa

RUN sudo apt update
RUN sudo apt install -y clang libssl-dev ninja-build wget make curl opam

# Install OCaml
RUN opam init -y --disable-sandboxing
RUN eval $(opam config env)
RUN opam install -y dune core menhir

# Install CMAKE
RUN wget https://github.com/Kitware/CMake/releases/download/v3.23.1/cmake-3.23.1.tar.gz
RUN tar -xf cmake-3.23.1.tar.gz
RUN cd cmake-3.23.1
RUN ./configure
RUN make
RUN make install

# Install Rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- --default-toolchain nightly -y
8 changes: 4 additions & 4 deletions .github/continuous-integration.sh
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ function run-mlir-tests {
run-step ninja -C $ARC_MLIR_BUILD/llvm-build/ check-arc-mlir
}

function run-runtime-tests {
cd arc-runtime
function run-lang-tests {
cd arc-lang
run-step arc-cargo "$@"
}

Expand Down Expand Up @@ -123,8 +123,8 @@ case $1 in
;;

cargo)
# We assume this is a arc-runtime cargo command line
# We assume this is a arc-sys cargo command line
shift
run-runtime-tests "$@"
run-sys-tests "$@"
;;
esac
2 changes: 1 addition & 1 deletion .github/workflows/arc-mlir-broken-links.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
steps:

- name: Checkout source
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: Update submodule
run: git submodule update --force --init --recursive
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ env:

jobs:
build-deploy:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest

steps:
- name: Checkout source
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: Update apt index
run: sudo apt-get update
Expand All @@ -34,28 +34,28 @@ jobs:

- name: Cache arc-preprocessor
id: cache-arc-preprocessor
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: arc-docs/arc-preprocessor/target
key: ${{ runner.os }}-cargo-${{ hashFiles('arc-docs/arc-preprocessor/**') }}

- name: Cache mdbook-bib
id: cache-mdbook-bib
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: ~/.cargo/bin/mdbook-bib
key: ${{ runner.os }}-cargo-${{ env.MDBOOK_BIB_VERSION }}

- name: Cache mdbook-linkcheck
id: cache-mdbook-linkcheck
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: ~/.cargo/bin/mdbook-linkcheck
key: ${{ runner.os }}-cargo-${{ env.MDBOOK_LINKCHECK_VERSION }}

- name: Cache zola
id: cache-zola
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: ~/.cargo/bin/zola
key: ${{ runner.os }}-cargo-${{ env.ZOLA_VERSION }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
# target: x86_64-apple-darwin
steps:
- name: Checkout source
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
submodules: 'recursive'

Expand All @@ -44,7 +44,7 @@ jobs:
mv README.md "$package"
mv arc-mlir/build/llvm-build/bin/{arc,arc-lang,arc-mlir} "$package/bin"
mv {arc-runtime,arc-lang/stdlib} "$package/share"
mv {arc-lang/stdlib} "$package/share"
tar cvf "$package.tar" "$package"
Expand Down
18 changes: 1 addition & 17 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ on:
paths:
- "arc-lang/**"
- "arc-mlir/**"
- "arc-runtime/**"
- "arc-python/**"
- "build"
- ".github/workflows/test.yml"
Expand All @@ -25,7 +24,7 @@ jobs:

# Checkout branch which we want to test
- name: Checkout source
uses: actions/checkout@v2
uses: actions/checkout@v3

# Update LLVM dependency
- name: Update submodule
Expand All @@ -41,29 +40,14 @@ jobs:
env:
ARC_MLIR_BUILD: ${{ format('{0}/build', runner.temp) }}

- name: Pre-build arc-runtime
run: ./.github/continuous-integration.sh cargo test
env:
ARC_MLIR_BUILD: ${{ format('{0}/build', runner.temp) }}

- name: Run MLIR tests
run: ./.github/continuous-integration.sh run-mlir-tests
env:
ARC_MLIR_BUILD: ${{ format('{0}/build', runner.temp) }}

- name: "Run arc-runtime test: clippy"
run: ./.github/continuous-integration.sh cargo clippy
env:
ARC_MLIR_BUILD: ${{ format('{0}/build', runner.temp) }}

- name: "Ccache/Sccache statistics"
run: ./.github/continuous-integration.sh check-ccache

# - name: "Run arc-runtime test: fmt -- -v --check"
# run: ./.github/continuous-integration.sh cargo fmt -- -v --check
# env:
# ARC_MLIR_BUILD: ${{ format('{0}/build', runner.temp) }}

- name: Check for dirty tree
run: ./check-for-dirty-tree

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,6 @@ book/
# zola

static/

# JavaScript
node_modules/
58 changes: 14 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,47 +11,23 @@ OCaml (and dune), Rust (and cargo), and C++ (and CMake and Ninja).

## Examples

A basic streaming word-count application can be written in functional-style as follows:
```
val wordcounts = lines
.flatmap(_.split(" "))
.keyby(_)
.window(
length = 10min,
stride = 3min
)
.count()
```

The same code can also be written using a more declarative, relational-style, syntax. This concept is borrowed from [Morel](https://github.com/julianhyde/morel) and applied to streaming data.
A streaming word-count application can be implemented in Arc-Lang as follows.

```
val wordcounts =
from
line in lines,
word in line.split(" ")
keyby word
window
length = 10min
stride = 3min
reduce count
identity 1;
def main() =
from line: String in source(topic: "text"),
word in line.split(" ") {
group word
window count as w {
length 10min
step 3min
compute count
}
select {word, w.count}
into sink(topic: "wordcount")
}
```

## Feature highlights

* Statically typed with global type inference.
* Parametric polymorphism (generics and rows) and ad-hoc polymorphism (type classes).
* Mix of functional syntax, imperative control-flow/mutation, and relational operators.
* Algebraic data types.
* First-class data streams.
* Complex event processing using tasks.
* Window-based computation.
* Low-level compilation and distributed execution.
* Command-line interface for data ingestion.

Note: All features have not yet been implemented :)

## Installation

```bash
Expand All @@ -71,13 +47,7 @@ git submodule update --init --recursive

* [`arc-lang`](https://github.com/cda-group/arc/tree/master/arc-lang) - A compiler for Arc-Lang.
* [`arc-mlir`](https://github.com/cda-group/arc/tree/master/arc-mlir) - An optimizer for Arc-Lang.
* [`arc-runtime`](https://github.com/cda-group/arc/tree/master/arc-runtime) - A local runtime which supports the execution of Arc-Lang programs.
* [`arc-python`](https://github.com/cda-group/arc/tree/master/arc-python) - A Python library for writing Arc-Lang applications.

## Related Projects

* [`arcon`](https://github.com/cda-group/arcon) - A distributed runtime which will support execution of Arc-Lang.
* [`kompact`](https://github.com/kompics/kompact) - A component-actor middleware which Arc-Runtime and Arcon are both implemented in.
* [`arc-sys`](https://github.com/cda-group/arc/tree/master/arc-sys) - A distributed system for executing Arc-Lang programs.

## Other

Expand Down
3 changes: 1 addition & 2 deletions arc-docs/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
all:
cargo doc --document-private-items --no-deps --target-dir=src/target --manifest-path=../arc-runtime/Cargo.toml
cargo doc --document-private-items --no-deps --target-dir=src/target --manifest-path=../arc-runtime/macros/Cargo.toml
cargo doc --document-private-items --no-deps --target-dir=src/target --manifest-path=../arc-sys/Cargo.toml
mdbook build

build:
Expand Down
Loading

0 comments on commit aa7f604

Please sign in to comment.