O365 --> MS365 - A potential big change - your views needed

[RogerSelwyn]

👋 Your views needed!

I'm using Discussions as a place to connect with other members of the community and understand your point of view on a potential major change.

I've noted in the release notes to v4.8.0 that the integration is becoming unwieldy. It has grown under my stewardship from an integration that supported calendars and email, to one that supports a whole variety of different sensors. This makes it slow to start up and operate and along with being a massive chunk of code to maintain.

I've started down a road that to breaking this up into smaller components, probably along the lines of Calendar, Email and Teams. To that end I have created a new integration just for calendars. I've branded this as MS365 inline with Microsoft's changed branding of Office 365 to Microsoft 365. This has the benefit of clearly separating the integrations.

I'm very conscious that there is a significant following of this integration, so this may be an unpopular change, but I feel it is necessary. Other may suggest doing it another way, and I'm certainly open to views. My knowledge of what can be done with Python and HA is a long way of perfect, I'm an enthusiastic amateur.

I've taken to decision to make in the installation of the integration as being UI only. The calendars.yaml file still exists and the calendars can be managed via that method, but there is the ability to modify most of the config via the UI.

Again, this may be an unpopular decision because many want to be yaml only. I have previously managed an integration that does both YAML and UI, but it is a pain, so I'm not doing it again. I think with recent advances in the HA UI around reconfigure and reload, this will remove the need to go delete tokens or reboot HA.

Anyway, I'm open to comments. For reference the new calendar is here MS365-Calendar. I would warn that it needs HA 2024.6.x since it incorporates changes included with that release.

[B-Hartley]

I only really use to-do
Where will that sit?

I have no strong feelings either way.

If it's easier for you to break it up. That's good enough for me.

[topgun0dk]

Same i do

[RogerSelwyn]

I have already created the new ToDo integration.

[topgun0dk]

I saw that further down. And it is now installed :)
Thanks a lot for your work

[RogerSelwyn]

Probably in Teams, but I may make that a fourth Integration. It’s quite complex like calendar.

[lexiismadd]

I think separated is a good idea, easier to manage, easier to diagnose issues.
Teams is definitely one area I use

[RogerSelwyn]

The dedicated Teams integration is available, there is a link in this thread further down

[SamBouwer]

@RogerSelwyn thank you for the open communication! I would fully support the split into separate integrations like you did for Calendar. It might be a (bit of a) pain for users, but I can see it benefits the maintenance of the integrations.

[uSlackr]

@RogerSelwyn I've been using this since you took it over and appreciate the long-term effort and commitment you've made to this project. I like the idea of breaking up the code to reduce the load on the system and the less code the fewer bugs affect any particular install. Are you concerned you could wind up making the same changes 3-4 times?
FWIW I use calendar and to-do.

[RogerSelwyn]

I am concerned about duplication in the initial, config_flow, permissions logic. I might break it out into generic/specific code blocks, so I can copy content between integrations. I'll figure that as I get into the second one. I may break ToDo into a fourth module since it is similar to calendar in concept.

[RogerSelwyn]

As a passing comment, I haven't taken advantage of HA application credentials, because in general that seems to assume it will request, hold and manage the token, and the python-o365 simply isn't structured to work like that.

[loryanstrant]

I do think it's logical to break it up, as realistically there are different use cases across the M365 stack, plus it allows more people to contribute to certain areas of passion (and hopefully for those who have created other "competing" components to pool their efforts into these ones).
While I know practically nothing of Python, I do work with M365 and Graph daily (it's literally my job to help orgs out with this stuff and build custom solutions on it), so if you like I'm happy to get more involved across all the various components (whether it's architecture, integration in M365, functionality, testing, documentation, etc.).

[RogerSelwyn]

Thank you very much for your offer, much appreciated. I'll do the split up, and try to document what it is doing as best I can in some form (given it goes via a 3rd party library). Then maybe you can have a look and see if there are better ways of doing what it does.

[smarthomefamilyverrips]

First of all thank you for the work you put in to this. To me it seems a good choice as looking at other comments here more people just use a part of the functionality of the O365 integration same as I do. I only need/use it for getting Calendar data of my Outlook Calendar from work, so I personally I prefer a less heavy integration giving me just what I need. So definitely a +1 from me on this.

I surely will try out the M365-Calendar integration.

[RogerSelwyn]

For those interested, I have started writing up some documentation on what calls are made as part of the main retrieval cycle - https://github.com/RogerSelwyn/O365-HomeAssistant/wiki

I need to think about repository structure, to see if I can store all the integrations in one repository rather than multiple. This will depend on what HACS supports.

[RogerSelwyn]

I need to think about repository structure, to see if I can store all the integrations in one repository rather than multiple. This will depend on what HACS supports.

Well that question is answered:

- There must only be one integration per repository, i.e. there can only be one subdirectory to ROOT_OF_THE_REPO/custom_components/. If there are more than one, only the first one will be managed.
- All files required for the integration to run must be located inside the directory ROOT_OF_THE_REPO/custom_components/INTEGRATION_NAME/.

[RogerSelwyn]

I've just released a first version of the MS365-ToDo integration. Again needs 2024.6.x

[B-Hartley]

It says I need 2024.6.0 to install it, but that version isn't out yet.
I'm on 2024.6.0b4

[B-Hartley]

I've installed manually for now, I'll let you know how it goes.

[RogerSelwyn]

I would have thought it would allow that. You can always manually copy it. Otherwise wait until Monday

[B-Hartley]

Installed manually, configured through UI. All seemed to go ok and my lists are showing. No time to test more right now, but looks ok ! Thanks

[RogerSelwyn]

I've just released a first version of the MS365-Mail integration. Again needs 2024.6.x

Teams is still to come, but won't get done until mid-July. I've also requested HACS inclusion for the three already done, but that will take some months to get included.

[RogerSelwyn]

I've just release a first version of MS365-Teams. I will submit a HACS inclusion request as soon as the brand icon has been accepted into HA Core.

Expect HACS to take many months, it is very, very slow.

[RogerSelwyn]

Just to update. HACS seems to have stopped accepting new integrations at the moment. I suspect this is because a new version is being developed which is currently in Alpha (or was when I looked a week ago). Hopefully when it is released they will work through the backlog.

I will endeavour to maintain both the O365 integration, and the split up versions until they make it onto HACS proper. I will then do some "marketing" on the new ones to encourage the move so I can decommission the old. This of course may take some time.

[tastyllama]

Thanks for the updates! Appreciate all your hard work.

[cromulus]

Amazing work. Thank you!!! IMHO, this should be HA core.

[ClearlyDazed]

Thank you for this amazing work. Splitting up into individual modules is an excellent way forward. Not everyone utilizes all the modules from Microsoft 365.
With the recent security upgrade by Microsoft to allow only Oauth2 your project seems to be a godsent to access Outlook 365 to send email notifications from Home Assistant. It also seems that Microsoft Azure has changed and your instructions on setting up Azure do not correlate. I was wondering if you could elaborate on what the new procedure would be to setup Azure.
Thanks

[RogerSelwyn]

To be honest I have no idea. Not sure I've ever received an email saying I must be on MFA by 14th October for access to Entra Portal (new name for Azure). Seems I might need a P1 license which is unlikely to ever happen, I only use it for email.

It might die from what I can see.

[RogerSelwyn]

It also seems that Microsoft Azure has changed and your instructions on setting up Azure do not correlate. I was wondering if you could elaborate on what the new procedure would be to setup Azure.

Not withstanding the MFA issue I mentioned, the create of the Entra/Azure app is exactly as described in the documentation. Perhaps you could expand on how you think it has changed?

[ClearlyDazed]

Your link to the portal initially sent me to an older site that required me to choose a site to register an account first. Everything else fell into place once I found the app registration option.

[RogerSelwyn]

Ok, I’ll check

[dreamworks]

I would love to chat/discuss with you about the O365 ./. M365 challenge.
(done more than 15 years of SharePoint O365/C#/REST development)

[RogerSelwyn]

Happy to have a chat some time. Give me a ping on firstname @ surname org uk

[pantherale0]

Hi @RogerSelwyn - some great work here and I use mainly the Teams side of things, but have an awful lot of experience with Graph API itself.

As we're using OAuth flows, shouldn't the integrations use Application Credentials in HA as documented here? https://developers.home-assistant.io/docs/core/platform/application_credentials/

As a side note - breaking this down will help me contribute to areas I use the most (Teams), so will probably start sending some PRs soon for new functionality.

[RogerSelwyn]

I’ve looked at using the HA app credentials mechanism, but really the library the integrations use (python-o365) isn’t set up to work with the ha mechanism. HA wants to do the oauth calls itself, whilst the library is designed to handle it all itself.

[RogerSelwyn]

Any prs would be appreciated, just give me an idea what you plan before you submit something please.

[fenty17]

@RogerSelwyn - can you advise on the best way to upgrade from O365 to MS365 Calendar. Can we install side by side or save any config, or is it set the new one up from scratch?

[RogerSelwyn]

You can install side by side. If you have done config in calendars.yaml (or todos/tasks.yaml) then you will be able to copy that content across to MS365 after you install the new integration. After you have disabled O365, you could then rename any entities to match the old ones, avoiding having to update automations/etc.

I'm not sure it would be easy for me to automate it, because one is in yaml and the other is via UI (which will avoid the endless mis-config issues I get), but I'll chew on it before we get into any forced migration in the middle of next year.

[RogerSelwyn]

Just to add, I'm working through create an o365 service that will create the config entries for the relevant MS365 integrations. All you will need to do is click on the integration option to re-configure and then re-authenticate to MS.