s3v2 seems to ignore input cfgs, using just defaults

[shlomi-dr]

Trying to use the new s3v2 apis, I started getting error messages that "I did not specify regions", though I did. Debugging lead me to the following, where the cfgs defined on line 71 are silently ignored by the body:

parquet-go-source/s3v2/s3.go

Lines 66 to 80 in b4f792c

	func NewS3FileWriter(
	ctx context.Context,
	bucket string,
	key string,
	uploaderOptions []func(*manager.Uploader),
	cfgs ...*aws.Config,
	) (source.ParquetFile, error) {
	return NewS3FileWriterWithClient(
	ctx,
	s3.NewFromConfig(getConfig()),
	bucket,
	key,
	uploaderOptions,
	)
	}

Am I doing something wrong?

[shlomi-dr]

Worked around this by using NewS3FileWriterWithClient directly.

Really confused why NewS3FileWriter has such misleading signature..

[sjtower]

The same problem exists for func NewS3FileReader:

// NewS3FileReader creates an S3 FileReader, to be used with NewParquetReader
func NewS3FileReader(ctx context.Context, bucket string, key string, cfgs ...*aws.Config) (source.ParquetFile, error) {
	return NewS3FileReaderWithClient(ctx, s3.NewFromConfig(getConfig()), bucket, key)
}

the input parameter cfgs is not used. Shouldn't there only be one configuration passed?

EG:

// NewS3FileReader creates an S3 FileReader, to be used with NewParquetReader
func NewS3FileReader(ctx context.Context, bucket string, key string, cfg aws.Config) (source.ParquetFile, error) {
	return NewS3FileReaderWithClient(ctx, s3.NewFromConfig(cfg), bucket, key)
}

[shlomi-dr]

Yeah, don't know why these functions take n cfgs, and use 0 🤷‍♂️

For now I just work with the *WithClient functions.