Integration with Apache Mina not working

[rashmimaharana]

Hi @carlspring

Bug Description

I have been trying to integrate this file system with Apache Mina sshd 2.12.1. But its failing. Here is the code snippet that I configured.
If I pass the bucket name as plain with out any / , its throwing error as this.fileStore is null and when i passed the bucket with /mybucket/, it's connecting but giving error as below. I am using org.carlspring.cloud.aws:s3fs-nio:1.0.5.

Steps To Reproduce

1. Setup Apache Mina sshd server with version 2.12.1.
2. Integrate with org.carlspring.cloud.aws:s3fs-nio:1.0.5.
3. Create a FileSystemFactory like below and integrate with Apache Mina 2.12.1.

sshd.setFileSystemFactory(createFileSystemFactory("mybucket"));

public FileSystemFactory createFileSystemFactory(String bucketName) throws IOException {
        Map<String, ?> env = ImmutableMap.<String, Object>builder().put(ACCESS_KEY, "AKIAX***")
                .put(SECRET_KEY, "hnBpn****")
                .put(REGION,"us-east-1")
                .build();
        FileSystem fileSystem = FileSystems.newFileSystem(URI.create("s3:///"),
                                                                    env,
                                                                    Thread.currentThread()
                                                                        .getContextClassLoader());
            Path bucketPath = fileSystem.getPath(bucketName);
            return new VirtualFileSystemFactory(bucketPath);
        }

Expected Behavior

Should be able to connect to s3 bucket and list files

Environment

• s3fs-nio version: s3fs-nio:1.0,5
• OS: Windows
• JDK: 17

  full output of mvn -version (or java -version)
  Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
  Maven home: C:\Users\rrmahar\Documents\binary\apache-maven-3.9.5
  Java version: 17.0.11, vendor: Azul Systems, Inc., runtime: C:\Program Files\Zulu\zulu-17
  Default locale: en_IN, platform encoding: Cp1252
  OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"
  
  java -version
  openjdk version "17.0.11" 2024-04-16 LTS
  OpenJDK Runtime Environment Zulu17.50+20-SA (build 17.0.11+9-LTS)
  OpenJDK 64-Bit Server VM Zulu17.50+20-SA (build 17.0.11+9-LTS, mixed mode, sharing)
  

Screenshots

Additional context

Exception 1

-nio2-thread-9] o.a.s.server.session.ServerSessionImpl   : exceptionCaught(ServerSessionImpl[rrmahar@/[0:0:0:0:0:0:0:1]:50619])[state=Opened] NullPointerException: Cannot invoke "org.carlspring.cloud.storage.s3fs.S3FileStore.name()" because "this.fileStore" is null

Exception 2

-nio2-thread-6] o.a.s.server.session.ServerSessionImpl   : exceptionCaught(ServerSessionImpl[rrmahar@/[0:0:0:0:0:0:0:1]:50662])[state=Opened] UnsupportedOperationException: Not a directory: s3://AKIAXMZ***@s3.amazonaws.com/mybucket/  

Proposed Solution

Known Workarounds

Useful Links

Task Relationships

This bug:

• Is caused by:
• Relates to:
  • Apache MINA integration #844
• Depends on:

[carlspring]

Hi @rashmimaharana @SergiyBojko,

Your bug reports seems to be related.

Would you guys be able to provide JUnit test cases for them? Perhaps something with TestContainers?

We would be really happy to receive pull requests with fixes for this as we have limited resources at the moment.

[steve-todorov]

@rashmimaharana We haven't had a chance to look into this yet. Can you try the code you have with the 1.0.6-SNAPSHOT version which is published in Maven Central Snapshots? There was a bug related to the properties not being passed from the Files.newFileSystem down to the S3FileSystem which could have been related to the problem you are seeing.

[rashmimaharana]

Hi @steve-todorov
Seems like issue was with my network where netskope was blocking any connection made to s3 object hosted outside the organization. It worked after disabling the netskope.
Any reference do you have where we can implement the filesystem using rest api where instead of using s3 sdk we will consume the rest api which are written on top s3 to enable file transfer.

[carlspring]

@rashmimaharana ,

I'm not sure what you're trying to ask here:

Any reference do you have where we can implement the filesystem using rest api where instead of using s3 sdk we will consume the rest api which are written on top s3 to enable file transfer.

The whole point of a library like this is to abstract you from the REST API so that you can use a more native implementation to the Java SDK. If you would like to use the AWS REST API directly, you can just write an HTTP client of your own to do so., although I really don't see how this will help.

[steve-todorov]

I am also not very sure what you are trying to do. As @carlspring explained -- this library allows you to use Java NIO and abstract away the interaction with the AWS S3 API. If you want to use the rest api -- then you can simply use the official AWS SDK v2 library which provides an S3 client with all the requests / responses you can use.

[rashmimaharana]

We can close this as its was blocked in my netskope. Thanks for response