Skip to content

Commit b109735

Browse files
stevenjbkioshndtscalac
authored
fix(cat-gateway): OpenAPI Lint Cleanup Part 2 (#1099)
* fix: api endpoint draft Signed-off-by: bkioshn <[email protected]> * fix: api health endpoint v1 Signed-off-by: bkioshn <[email protected]> * fix: remove bad request from errorResponses Signed-off-by: bkioshn <[email protected]> * fix: add bad req to get /registration Signed-off-by: bkioshn <[email protected]> * fix: error logging Signed-off-by: bkioshn <[email protected]> * fix: remove validation error Signed-off-by: bkioshn <[email protected]> * fix: registration get error name Signed-off-by: bkioshn <[email protected]> * chore:format Signed-off-by: bkioshn <[email protected]> * fix: get json schema from openapi spec Signed-off-by: bkioshn <[email protected]> * fix: move schema utils Signed-off-by: bkioshn <[email protected]> * fix: optional field Signed-off-by: bkioshn <[email protected]> * fix: config key Signed-off-by: bkioshn <[email protected]> * fix: cat-gateway code gen Signed-off-by: bkioshn <[email protected]> * fix: api name in cat-voice Signed-off-by: bkioshn <[email protected]> * fix: cat-voice format Signed-off-by: bkioshn <[email protected]> * chore: fix spacing Signed-off-by: bkioshn <[email protected]> * chore: fix spacing Signed-off-by: bkioshn <[email protected]> * chore: change tag config description * test: add test for default validator * fix: add spectral ruleset Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): Sort the spelling words, and use latest deny.toml * fix(cat-gateway): Fix broken pre-push justfile target * docs(cat-gateway): cleanup * docs(cat-gateway): Fix API Groups and document them better * docs(cat-gateway): Add documentation to the health/inspection endpoint * docs(cat-gateway): Add descriptions for cardano/cip36/latest_registration/stake_addr * docs(cat-gateway): Document stake key hash and vote key endpoints for cardano * docs(cat-gateway): add documentation to config/frontend * docs(cat-gateway): Add api docs for frontend schema * docs(cat-gateway): Move legacy registration endpoints into the Legacy TAG. * docs(cat-gateway): Remaining documentable entities documented * fix: update openapi linter Signed-off-by: bkioshn <[email protected]> * docs(cat-gateway): Add more constraints to parameters and json bodies * fix: openapi lint FUNCTION name Signed-off-by: bkioshn <[email protected]> * fix: CIP36 example and description Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): cleanup error handling, and add a global 429 response to all endpoints. * fix: config endpoint example, desc, and return Signed-off-by: bkioshn <[email protected]> * chore: remove todo Signed-off-by: bkioshn <[email protected]> * fix: move config object Signed-off-by: bkioshn <[email protected]> * fix: move cip36 object Signed-off-by: bkioshn <[email protected]> * docs(cat-gateway): Add missing headers to responses * docs(cat-gateway): Cleanup the rest of the documentation in the api * fix(cat-gateway): Fix OpenAPI linting and add autogenerated api file for dart. * refactor(cat-gateway): Better generalize the OpenAPI simple string type creation macro. * fix(cat-gateway): Add APIKey and CatToken auth to some endpoints. Add 401 and 403 common responses. * fix(cat-gateway): Add universal 422 response to all endpoints, and try and make all endpoint validation use it. * fix: add cardano stake address type Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): stake address type Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): Refactor the RBAC Token auth, so it's easier to maintain. * fix(cat-gateway): stake address name Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): Add no auth and no-auth+rbac auth schemes * fix(cat-gateway): format + stake addr example Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): code format * fix(cat-gateway): openapi spectral example rules Signed-off-by: bkioshn <[email protected]> * fix(cat-gateway): Move legacy registration endpoint under Legacy Tag * fix(cat-gateway): Add Auth to all endpoints * fix(docs): Remove obsolete lint config file * fix(cat-gateway): Make config.toml match upstream * docs(docs): update project dictionary * feat(cat-gateway): add target to make it quick to check openapi lints locally * fix(cat-gateway): Remove reference to hermes * fix(cat-gateway): Add auth to rbac endpoints * docs(cat-gateway): Add full docs for v1/votes/plan/account-votes * docs(cat-gateway): Add example for ip address query argument * fix(cat-gateway): Define and abstract Ed25519 Public Keys as hex encoded parameters * fix(cat-gateway): Make sure string api types do not directly expose the internal string * fix(cat-gateway): Make conversion from a Ed25519 pub key hex value to a Verifyingkey infallible * fix(cat-gateway): Fix native asset response types * docs(cat-gateway): fix comments * fix(cat-gateway): Autogenerate flutter files * fix(cat-gateway): Exclude legacy endpoints from needing api examples --------- Signed-off-by: bkioshn <[email protected]> Co-authored-by: bkioshn <[email protected]> Co-authored-by: bkioshn <[email protected]> Co-authored-by: Dominik Toton <[email protected]>
1 parent a08cbed commit b109735

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+961
-475
lines changed

.config/dictionaries/project.dic

+3
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ COCOAPODS
5353
codegen
5454
codepoints
5555
commitlog
56+
concatcp
5657
coti
5758
coverallsapp
5859
CQLSH
@@ -184,6 +185,7 @@ netifas
184185
netkey
185186
nextest
186187
Nodetool
188+
oapi
187189
OCSP
188190
Oleksandr
189191
onboarded
@@ -230,6 +232,7 @@ ristretto
230232
rlib
231233
rngs
232234
RPATH
235+
ruleset
233236
rustc
234237
rustdoc
235238
rustdocflags

catalyst-gateway/Justfile

+6-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# use with https://github.com/casey/just
22
#
3-
# Hermes developer convenience functions
3+
# Developer convenience functions
44

55
# cspell: words prereqs, commitlog, rustls, nocapture
66

@@ -56,3 +56,8 @@ run-cat-gateway-mainnet: build-cat-gateway
5656
CHAIN_FOLLOWER_DL_CHUNK_SIZE="4" \
5757
RUST_LOG="error,cat_gateway=debug,cardano_chain_follower=debug,mithril-client=debug" \
5858
./target/release/cat-gateway run --log-level debug
59+
60+
# Do the minimal work needed to test the schema generated by cat-gateway
61+
quick-schema-lint: build-cat-gateway
62+
./target/release/cat-gateway docs cat-gateway-api.json
63+
docker run --rm -it -v $(pwd):/tmp stoplight/spectral:latest lint --ruleset "/tmp/tests/.oapi-v3.spectral.yml" "/tmp/cat-gateway-api.json"

catalyst-gateway/bin/Cargo.toml

+17-19
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ pallas = { version = "0.30.1", git = "https://github.com/input-output-hk/catalys
2222
pallas-traverse = { version = "0.30.1", git = "https://github.com/input-output-hk/catalyst-pallas.git", rev = "9b5183c8b90b90fe2cc319d986e933e9518957b3" }
2323
#pallas-crypto = { version = "0.30.1", git = "https://github.com/input-output-hk/catalyst-pallas.git", rev = "9b5183c8b90b90fe2cc319d986e933e9518957b3" }
2424

25-
clap = { version = "4.5.18", features = ["derive", "env"] }
25+
clap = { version = "4.5.20", features = ["derive", "env"] }
2626
tracing = { version = "0.1.40", features = ["log"] }
2727
tracing-subscriber = { version = "0.3.18", features = [
2828
"fmt",
@@ -32,30 +32,26 @@ tracing-subscriber = { version = "0.3.18", features = [
3232
"time",
3333
"env-filter",
3434
] }
35-
serde = { version = "1.0.210", features = ["derive"] }
36-
serde_json = "1.0.128"
37-
thiserror = "1.0.64"
35+
serde = { version = "1.0.214", features = ["derive"] }
36+
serde_json = { version = "1.0.132", features = ["arbitrary_precision"] }
37+
thiserror = "1.0.68"
3838
chrono = "0.4.38"
39-
# async-trait = "0.1.82"
40-
bb8 = "0.8.5"
39+
bb8 = "0.8.6"
4140
bb8-postgres = "0.8.1"
4241
tokio-postgres = { version = "0.7.12", features = [
4342
"with-chrono-0_4",
4443
"with-serde_json-1",
4544
"with-time-0_3",
4645
] }
47-
tokio = { version = "1.40.0", features = ["rt", "macros", "rt-multi-thread"] }
46+
tokio = { version = "1.41.0", features = ["rt", "macros", "rt-multi-thread"] }
4847
dotenvy = "0.15.7"
4948
local-ip-address = "0.6.3"
5049
gethostname = "0.5.0"
5150
hex = "0.4.3"
52-
handlebars = "6.1.0"
53-
anyhow = "1.0.89"
54-
#cddl = "0.9.4"
55-
#ciborium = "0.2.2"
56-
# stringzilla = "3.9.3"
51+
handlebars = "6.2.0"
52+
anyhow = "1.0.92"
5753
duration-string = "0.4.0"
58-
build-info = "0.0.38"
54+
build-info = "0.0.39"
5955
ed25519-dalek = "2.1.1"
6056
scylla = { version = "0.14.0", features = ["cloud", "full-serialization"] }
6157
strum = { version = "0.26.3", features = ["derive"] }
@@ -70,8 +66,8 @@ rust_decimal = { version = "1.36.0", features = [
7066
"serde-with-float",
7167
"db-tokio-postgres",
7268
] }
73-
poem = { version = "3.1.0", features = ["embed", "prometheus", "compression"] }
74-
poem-openapi = { version = "5.1.1", features = [
69+
poem = { version = "3.1.3", features = ["embed", "prometheus", "compression"] }
70+
poem-openapi = { version = "5.1.2", features = [
7571
"openapi-explorer",
7672
"rapidoc",
7773
"redoc",
@@ -80,10 +76,10 @@ poem-openapi = { version = "5.1.1", features = [
8076
"url",
8177
"chrono",
8278
] }
83-
uuid = { version = "1.10.0", features = ["v4", "serde"] }
79+
uuid = { version = "1.11.0", features = ["v4", "serde"] }
8480
ulid = { version = "1.1.3", features = ["serde", "uuid"] }
8581
blake2b_simd = "1.0.2"
86-
url = "2.5.2"
82+
url = "2.5.3"
8783
panic-message = "0.3.0"
8884
cpu-time = "1.0.0"
8985
prometheus = "0.13.4"
@@ -93,10 +89,12 @@ base64 = "0.22.1"
9389
dashmap = "6.1.0"
9490
x509-cert = "0.2.5"
9591
der-parser = "9.0.0"
96-
jsonschema = "0.22.3"
92+
jsonschema = "0.26.1"
93+
bech32 = "0.11.0"
94+
const_format = "0.2.33"
9795

9896
[dev-dependencies]
9997
proptest = "1.5.0"
10098

10199
[build-dependencies]
102-
build-info-build = "0.0.38"
100+
build-info-build = "0.0.39"

catalyst-gateway/bin/src/build_info.rs

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
//! Hermes binary build info
1+
//! Binary build info
22
33
use build_info::{self as build_info_crate};
44
use local_ip_address::list_afinet_netifas;
55
use tracing::info;
66

77
use crate::service::utilities;
88

9-
/// Formatted hermes binary build info
9+
/// Formatted Binary build info
1010
pub(crate) const BUILD_INFO: &str = build_info_crate::format!("
1111
version: {},
1212
git info: {{{}}}

catalyst-gateway/bin/src/db/index/block/certs.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ pub(crate) struct StakeRegistrationInsertQuery {
2525
slot_no: num_bigint::BigInt,
2626
/// Transaction Index.
2727
txn: i16,
28-
/// Full Stake Address (not hashed, 32 byte ED25519 Public key).
28+
/// Full Stake Public Key (32 byte Ed25519 Public key, not hashed).
2929
stake_address: MaybeUnset<Vec<u8>>,
3030
/// Is the stake address a script or not.
3131
script: bool,

catalyst-gateway/bin/src/db/index/queries/registrations/get_from_stake_addr.rs

+4-5
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,10 @@ pub(crate) struct GetRegistrationParams {
2424
pub stake_address: Vec<u8>,
2525
}
2626

27-
impl GetRegistrationParams {
28-
/// Create a new instance of [`GetRegistrationParams`]
29-
pub(crate) fn new(stake_addr: Vec<u8>) -> GetRegistrationParams {
30-
Self {
31-
stake_address: stake_addr,
27+
impl From<&ed25519_dalek::VerifyingKey> for GetRegistrationParams {
28+
fn from(value: &ed25519_dalek::VerifyingKey) -> Self {
29+
GetRegistrationParams {
30+
stake_address: value.as_bytes().to_vec(),
3231
}
3332
}
3433
}

catalyst-gateway/bin/src/db/index/schema/cql/txo_assets_by_stake_table.cql

+3-2
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@ CREATE TABLE IF NOT EXISTS txo_assets_by_stake (
77
txn smallint, -- Which Transaction in the Slot is the TXO.
88
txo smallint, -- offset in the txo list of the transaction the txo is in.
99
policy_id blob, -- asset policy hash (id) (28 byte binary hash)
10-
policy_name text, -- name of the policy (UTF8)
10+
policy_name text, -- name of the policy (UTF8) TODO: https://github.com/input-output-hk/catalyst-voices/issues/1121
11+
1112

1213
-- None Key Data of the asset.
13-
value varint, -- Value of the asset (u64)
14+
value varint, -- Value of the asset (i128)
1415

1516
PRIMARY KEY (stake_address, slot_no, txn, txo, policy_id, policy_name)
1617
);

catalyst-gateway/bin/src/service/api/cardano/cip36.rs

+51-41
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,15 @@
22
33
use std::{cmp::Reverse, sync::Arc};
44

5+
use anyhow::anyhow;
56
use futures::StreamExt;
67
use poem_openapi::{payload::Json, ApiResponse};
78
use tracing::error;
89

910
use crate::{
1011
db::index::{
1112
queries::registrations::{
12-
get_from_stake_addr::{GetRegistrationParams, GetRegistrationQuery},
13+
get_from_stake_addr::GetRegistrationQuery,
1314
get_from_stake_hash::{GetStakeAddrParams, GetStakeAddrQuery},
1415
get_from_vote_key::{GetStakeAddrFromVoteKeyParams, GetStakeAddrFromVoteKeyQuery},
1516
get_invalid::{GetInvalidRegistrationParams, GetInvalidRegistrationQuery},
@@ -21,7 +22,9 @@ use crate::{
2122
Cip36Info, Cip36Reporting, Cip36ReportingList, InvalidRegistrationsReport,
2223
},
2324
responses::WithErrorResponses,
25+
types::headers::retry_after::RetryAfterOption,
2426
},
27+
utils::ed25519,
2528
};
2629

2730
/// Endpoint responses.
@@ -51,18 +54,10 @@ pub(crate) type SingleRegistrationResponse = WithErrorResponses<ResponseSingleRe
5154
/// All responses voting key
5255
pub(crate) type MultipleRegistrationResponse = WithErrorResponses<ResponseMultipleRegistrations>;
5356

54-
/// Get latest registration given a stake address
57+
/// Get latest registration given a stake public key
5558
pub(crate) async fn get_latest_registration_from_stake_addr(
56-
stake_addr: String, persistent: bool,
59+
stake_pub_key: &ed25519_dalek::VerifyingKey, persistent: bool,
5760
) -> SingleRegistrationResponse {
58-
let stake_addr = match hex::decode(stake_addr) {
59-
Ok(stake_addr) => stake_addr,
60-
Err(err) => {
61-
error!(id="get_latest_registration_from_stake_addr", error=?err, "Failed to decode stake addr");
62-
return ResponseSingleRegistration::NotFound.into();
63-
},
64-
};
65-
6661
let Some(session) = CassandraSession::get(persistent) else {
6762
error!(
6863
id = "get_latest_registration_from_stake_addr",
@@ -72,7 +67,7 @@ pub(crate) async fn get_latest_registration_from_stake_addr(
7267
};
7368

7469
let registration =
75-
match latest_registration_from_stake_addr(stake_addr.clone(), session.clone()).await {
70+
match latest_registration_from_stake_addr(stake_pub_key, session.clone()).await {
7671
Ok(registrations) => registrations,
7772
Err(err) => {
7873
error!(
@@ -84,13 +79,10 @@ pub(crate) async fn get_latest_registration_from_stake_addr(
8479
},
8580
};
8681

87-
let invalids_report = match get_invalid_registrations(
88-
registration.stake_address.clone(),
89-
registration.slot_no.into(),
90-
session,
91-
)
92-
.await
93-
{
82+
let raw_invalids =
83+
get_invalid_registrations(stake_pub_key, registration.slot_no.into(), session).await;
84+
85+
let invalids_report = match raw_invalids {
9486
Ok(invalids) => invalids,
9587
Err(err) => {
9688
error!(
@@ -109,17 +101,19 @@ pub(crate) async fn get_latest_registration_from_stake_addr(
109101

110102
/// Get latest registration given a stake addr
111103
async fn latest_registration_from_stake_addr(
112-
stake_addr: Vec<u8>, session: Arc<CassandraSession>,
104+
stake_pub_key: &ed25519_dalek::VerifyingKey, session: Arc<CassandraSession>,
113105
) -> anyhow::Result<Cip36Info> {
114-
sort_latest_registration(get_all_registrations_from_stake_addr(session, stake_addr).await?)
106+
sort_latest_registration(
107+
get_all_registrations_from_stake_pub_key(session, stake_pub_key).await?,
108+
)
115109
}
116110

117111
/// Get all cip36 registrations for a given stake address.
118-
async fn get_all_registrations_from_stake_addr(
119-
session: Arc<CassandraSession>, stake_addr: Vec<u8>,
112+
async fn get_all_registrations_from_stake_pub_key(
113+
session: Arc<CassandraSession>, stake_pub_key: &ed25519_dalek::VerifyingKey,
120114
) -> Result<Vec<Cip36Info>, anyhow::Error> {
121115
let mut registrations_iter =
122-
GetRegistrationQuery::execute(&session, GetRegistrationParams::new(stake_addr)).await?;
116+
GetRegistrationQuery::execute(&session, stake_pub_key.into()).await?;
123117
let mut registrations = Vec::new();
124118
while let Some(row) = registrations_iter.next().await {
125119
let row = row?;
@@ -137,7 +131,7 @@ async fn get_all_registrations_from_stake_addr(
137131
};
138132

139133
let cip36 = Cip36Info {
140-
stake_address: hex::encode(row.stake_address),
134+
stake_pub_key: row.stake_address.try_into()?,
141135
nonce,
142136
slot_no,
143137
txn: row.txn,
@@ -162,13 +156,12 @@ fn sort_latest_registration(mut registrations: Vec<Cip36Info>) -> anyhow::Result
162156

163157
/// Get invalid registrations for stake addr after given slot no
164158
async fn get_invalid_registrations(
165-
stake_addr: String, slot_no: num_bigint::BigInt, session: Arc<CassandraSession>,
159+
stake_pub_key: &ed25519_dalek::VerifyingKey, slot_no: num_bigint::BigInt,
160+
session: Arc<CassandraSession>,
166161
) -> anyhow::Result<Vec<InvalidRegistrationsReport>> {
167-
let stake_addr = hex::decode(stake_addr)?;
168-
169162
let mut invalid_registrations_iter = GetInvalidRegistrationQuery::execute(
170163
&session,
171-
GetInvalidRegistrationParams::new(stake_addr, slot_no),
164+
GetInvalidRegistrationParams::new(stake_pub_key.as_bytes().to_vec(), slot_no),
172165
)
173166
.await?;
174167
let mut invalid_registrations = Vec::new();
@@ -177,7 +170,7 @@ async fn get_invalid_registrations(
177170

178171
invalid_registrations.push(InvalidRegistrationsReport {
179172
error_report: row.error_report,
180-
stake_address: hex::encode(row.stake_address),
173+
stake_address: row.stake_address.try_into()?,
181174
vote_key: hex::encode(row.vote_key),
182175
payment_address: hex::encode(row.payment_address),
183176
is_payable: row.is_payable,
@@ -212,11 +205,9 @@ pub(crate) async fn get_latest_registration_from_stake_key_hash(
212205
};
213206

214207
let Some(session) = CassandraSession::get(persistent) else {
215-
error!(
216-
id = "get_latest_registration_from_stake_key_hash_db_session",
217-
"Failed to acquire db session"
218-
);
219-
return ResponseSingleRegistration::NotFound.into();
208+
error!("Failed to acquire db session");
209+
let err = anyhow::anyhow!("Failed to acquire db session");
210+
return SingleRegistrationResponse::service_unavailable(&err, RetryAfterOption::Default);
220211
};
221212

222213
// Get stake addr associated with give stake hash
@@ -246,8 +237,17 @@ pub(crate) async fn get_latest_registration_from_stake_key_hash(
246237
},
247238
};
248239

240+
let stake_pub_key = match ed25519::verifying_key_from_vec(&row.stake_address) {
241+
Ok(v) => v,
242+
Err(err) => {
243+
error!(error=?err, "Invalid Stake Public Key in database.");
244+
let err = anyhow!(err);
245+
return SingleRegistrationResponse::internal_error(&err);
246+
},
247+
};
248+
249249
let registration = match latest_registration_from_stake_addr(
250-
row.stake_address.clone(),
250+
&stake_pub_key,
251251
session.clone(),
252252
)
253253
.await
@@ -266,7 +266,7 @@ pub(crate) async fn get_latest_registration_from_stake_key_hash(
266266
// include any erroneous registrations which occur AFTER the slot# of the last valid
267267
// registration
268268
let invalids_report = match get_invalid_registrations(
269-
registration.stake_address.clone(),
269+
&stake_pub_key,
270270
registration.slot_no.into(),
271271
session,
272272
)
@@ -347,12 +347,22 @@ pub(crate) async fn get_associated_vote_key_registrations(
347347
},
348348
};
349349

350+
let stake_pub_key = match ed25519::verifying_key_from_vec(&row.stake_address) {
351+
Ok(k) => k,
352+
Err(err) => {
353+
error!(
354+
id="get_associated_vote_key_registrations_latest_registration",
355+
error=?err,
356+
"Not a valid staking public key"
357+
);
358+
return ResponseMultipleRegistrations::NotFound.into();
359+
},
360+
};
361+
350362
// We have the stake addr associated with vote key, now get all registrations with the
351363
// stake addr.
352364
let registrations =
353-
match get_all_registrations_from_stake_addr(session.clone(), row.stake_address.clone())
354-
.await
355-
{
365+
match get_all_registrations_from_stake_pub_key(session.clone(), &stake_pub_key).await {
356366
Ok(registration) => registration,
357367
Err(err) => {
358368
error!(
@@ -375,7 +385,7 @@ pub(crate) async fn get_associated_vote_key_registrations(
375385

376386
for registration in redacted_registrations {
377387
let invalids_report = match get_invalid_registrations(
378-
registration.stake_address.clone(),
388+
&stake_pub_key,
379389
registration.slot_no.into(),
380390
session.clone(),
381391
)

0 commit comments

Comments
 (0)