forked from facebookincubator/velox
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Updates to README file (facebookincubator#11349)
Summary: Pull Request resolved: facebookincubator#11349 Some updates to the README file to reflect out current messaging and make it read more natural. Also pulling the community sections up. Reviewed By: xiaoxmeng, amitkdutta, bikramSingh91 Differential Revision: D64944089 fbshipit-source-id: 5ac64b8ce20cb09ce2f91d33acfeae64e7686cbc
- Loading branch information
1 parent
817fa90
commit b2c7446
Showing
3 changed files
with
71 additions
and
58 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,10 +17,13 @@ you are expected to uphold this code. | |
## Community | ||
|
||
A good first step to getting involved in the Velox project is to participate in | ||
conversations in GitHub [Issues](https://github.com/facebookincubator/velox/issues) | ||
and [Discussions](https://github.com/facebookincubator/velox/discussions), and join the | ||
[the Velox-OSS Slack workspace](http://velox-oss.slack.com) - please reach out to | ||
**[email protected]** to get access. | ||
conversations in GitHub | ||
[Issues](https://github.com/facebookincubator/velox/issues) and | ||
[Discussions](https://github.com/facebookincubator/velox/discussions), and join | ||
the [the Velox-OSS Slack workspace](http://velox-oss.slack.com) - please | ||
comment on [this | ||
Discussion](https://github.com/facebookincubator/velox/discussions/11348) to | ||
get access. | ||
|
||
## Components and Maintainers | ||
|
||
|
@@ -56,10 +59,11 @@ maintainers, allowing them to provide more timely feedback and keeping the | |
amount of rework from contributors to a minimum. | ||
|
||
We encourage new contributors to start with bug fixes and small features so you | ||
get familiar with the contributing process, while building relationships with | ||
community members. | ||
Look for GitHub issues labeled [good first issue](https://github.com/facebookincubator/velox/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or consider adding one of the | ||
[missing Presto SQL functions](https://github.com/facebookincubator/velox/issues/2262). | ||
get familiar with the contribution process, while building relationships with | ||
community members. Look for GitHub issues labeled [good first | ||
issue](https://github.com/facebookincubator/velox/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) | ||
or consider adding one of the [missing Presto SQL | ||
functions](https://github.com/facebookincubator/velox/issues/2262). | ||
|
||
The contribution process is outlined below: | ||
|
||
|
@@ -76,7 +80,7 @@ The contribution process is outlined below: | |
* Will the change conflict with another change in progress? If so, work with others to minimize impact. | ||
|
||
4. Implement the change. | ||
* Always follow the coding best practices outlined in the list below. | ||
* Always follow the [coding best practices](#coding-best-practices) outlined below. | ||
* If the change is large, consider posting a draft Github pull request (PR) | ||
with the title prefixed with [WIP], and share with collaborators to get early feedback. | ||
* Give the PR a clear, brief description; when the PR is | ||
|
@@ -88,9 +92,9 @@ The contribution process is outlined below: | |
* Create/submit a Github PR and tag the reviewers identified in Step 3. | ||
|
||
5. Review is performed by one or more reviewers. | ||
* This normally happens within a few days, but may take longer if the change is | ||
large, complex, or if a critical reviewer is unavailable (feel free to ping in the | ||
PR). | ||
* This normally happens within a few days, but may take longer if the change | ||
is large, complex, or if a critical reviewer is unavailable (feel free to | ||
ping them in the PR or on Slack). | ||
|
||
6. Address feedback and update the PR. | ||
* After pushing changes, add a comment to the PR mentioning the | ||
|
@@ -102,7 +106,7 @@ The contribution process is outlined below: | |
addressed and resolve the conversation. | ||
|
||
7. Iterate on this process until your changes are reviewed and accepted by a | ||
maintainer. At this point, a Meta employee will be required to merge your PR, | ||
maintainer. At this point, a Meta employee will be notified to merge your PR, | ||
due to tooling limitations. | ||
|
||
## Coding Best Practices | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,12 @@ | ||
<img src="static/logo.svg" alt="Velox logo" width="50%" align="center" /> | ||
|
||
Velox is a C++ database acceleration library which provides reusable, | ||
extensible, and high-performance data processing components. These components | ||
can be reused to build compute engines focused on different analytical | ||
workloads, including batch, interactive, stream processing, and AI/ML. | ||
Velox was created by Meta and it is currently developed in partnership with | ||
IBM/Ahana, Intel, Voltron Data, Microsoft, ByteDance and many other companies. | ||
Velox is a composable execution engine distributed as an open source C++ | ||
library. It provides reusable, extensible, and high-performance data processing | ||
components that can be (re-)used to build data management systems focused on | ||
different analytical workloads, including batch, interactive, stream | ||
processing, and AI/ML. Velox was created by Meta and it is currently developed | ||
in partnership with IBM/Ahana, Intel, Voltron Data, Microsoft, ByteDance and | ||
many other companies. | ||
|
||
In common usage scenarios, Velox takes a fully optimized query plan as input | ||
and performs the described computation. Considering Velox does not provide a | ||
|
@@ -16,25 +17,25 @@ integrating and optimizing their compute engines. | |
Velox provides the following high-level components: | ||
|
||
* **Type**: a generic typing system that supports scalar, complex, and nested | ||
types, such as structs, maps, arrays, tensors, etc. | ||
types, such as structs, maps, arrays, etc. | ||
* **Vector**: an [Arrow-compatible columnar memory layout | ||
module](https://facebookincubator.github.io/velox/develop/vectors.html), | ||
which provides multiple encodings, such as Flat, Dictionary, Constant, | ||
Sequence/RLE, and Bias, in addition to a lazy materialization pattern and | ||
support for out-of-order writes. | ||
providing encodings such as Flat, Dictionary, Constant, and Sequence/RLE, in | ||
addition to a lazy materialization pattern and support for out-of-order | ||
writes. | ||
* **Expression Eval**: a [fully vectorized expression evaluation | ||
engine](https://facebookincubator.github.io/velox/develop/expression-evaluation.html) | ||
that allows expressions to be efficiently executed on top of Vector/Arrow | ||
encoded data. | ||
* **Function Packages**: sets of vectorized function implementations following | ||
the Presto and Spark semantic. | ||
* **Operators**: implementation of common data processing operators such as | ||
scans, projection, filtering, groupBy, orderBy, shuffle, [hash | ||
join](https://facebookincubator.github.io/velox/develop/joins.html), unnest, | ||
and more. | ||
* **I/O**: a generic connector interface that allows different file formats | ||
(ORC/DWRF and Parquet) and storage adapters (S3, HDFS, local files) to be | ||
used. | ||
* **Functions**: sets of vectorized scalar, aggregates, and window functions | ||
implementations following the Presto and Spark semantic. | ||
* **Operators**: implementation of relational operators such as scans, writes, | ||
projections, filtering, grouping, ordering, shuffle/exchange, [hash, merge, | ||
and nested loop joins](https://facebookincubator.github.io/velox/develop/joins.html), | ||
unnest, and more. | ||
* **I/O**: a connector interface for extensible data sources and sinks, | ||
supporting different file formats (ORC/DWRF, Parquet, Nimble), and storage | ||
adapters (S3, HDFS, GCS, ABFS, local files) to be used. | ||
* **Network Serializers**: an interface where different wire protocols can be | ||
implemented, used for network communication, supporting | ||
[PrestoPage](https://prestodb.io/docs/current/develop/serialized-page.html) | ||
|
@@ -51,10 +52,11 @@ specializations, including: | |
1. Custom types | ||
2. [Simple and vectorized functions](https://facebookincubator.github.io/velox/develop/scalar-functions.html) | ||
3. [Aggregate functions](https://facebookincubator.github.io/velox/develop/aggregate-functions.html) | ||
4. Operators | ||
5. File formats | ||
6. Storage adapters | ||
7. Network serializers | ||
4. Window functions | ||
5. Operators | ||
6. File formats | ||
7. Storage adapters | ||
8. Network serializers | ||
|
||
## Examples | ||
|
||
|
@@ -68,6 +70,34 @@ of available functions [can be found here.](https://facebookincubator.github.io/ | |
|
||
Blog posts are available [here](https://velox-lib.io/blog). | ||
|
||
## Community | ||
|
||
Velox is an open source project supported by a community of individual | ||
contributors and organizations. The project's technical governance mechanics is | ||
described [in this | ||
document.](https://velox-lib.io/docs/community/technical-governance). | ||
|
||
Project maintainers [are listed | ||
here](https://velox-lib.io/docs/community/components-and-maintainers). | ||
|
||
The main communication channel with the Velox OSS community is through the [the | ||
Velox-OSS Slack workspace](http://velox-oss.slack.com), github Issues, and | ||
Discussions. | ||
|
||
For access to the Velox Slack workspace, please add a comment [to this | ||
Discussion](https://github.com/facebookincubator/velox/discussions/11348) | ||
|
||
## Contributing | ||
|
||
Check our [contributing guide](CONTRIBUTING.md) to learn about how to | ||
contribute to the project. | ||
|
||
## License | ||
|
||
Velox is licensed under the Apache 2.0 License. A copy of the license | ||
[can be found here.](LICENSE) | ||
|
||
|
||
## Getting Started | ||
|
||
### Get the Velox Source | ||
|
@@ -201,24 +231,3 @@ you can override the `NUM_THREADS` environment variable by doing: | |
```shell | ||
$ docker-compose run -e NUM_THREADS=<NUM_THREADS_TO_USE> --rm ubuntu-cpp | ||
``` | ||
|
||
## Contributing | ||
|
||
Check our [contributing guide](CONTRIBUTING.md) to learn about how to | ||
contribute to the project. | ||
|
||
## Community | ||
|
||
Velox's technical governance mechanics is described [in this | ||
document.](https://velox-lib.io/docs/community/technical-governance). | ||
Components and maintainers [are listed | ||
here](https://velox-lib.io/docs/community/components-and-maintainers). | ||
|
||
The main communication channel with the Velox OSS community is through the | ||
[the Velox-OSS Slack workspace](http://velox-oss.slack.com). | ||
Please reach out to **[email protected]** to get access to Velox Slack Channel. | ||
|
||
## License | ||
|
||
Velox is licensed under the Apache 2.0 License. A copy of the license | ||
[can be found here.](LICENSE) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,7 +18,7 @@ knowledge. All individuals listed in this page are expected to uphold | |
|
||
## Project Leadership Council - PLC | ||
|
||
The PLC is reponsible for the long-term directional leadership of the project: | ||
The PLC is responsible for the long-term directional leadership of the project: | ||
|
||
* Masha Basmanova - [mbasmanova](https://github.com/mbasmanova) / [email protected] - **Chair** | ||
* Orri Erling - [oerling](https://github.com/oerling) / [email protected] | ||
|