walshamsolutions

Microsoft Endpoint Manager: In-Place Upgrade or Side by Side Part 4 - SQL Server Requirement Analysis

dwalsham@walshamsolutions.com (D Walsham) — Mon, 13 Dec 2021 16:13:57 GMT

Looking through the current SQL Server topology and how it affects our decision

Introduction

In this part we are going to focus primarily around the SQL server aspect in our decision making.

Taking the time to understand how your SQL server environment fits to not only just the upgrade task at hand but also the overall future of future releases plays a big part. Many moving parts when it comes to your SQL Server architecture so this part we will go into fine detail in all parts around this topic and how they contribute towards your decision of its state and whether it does make sense to either proceed with an in-place upgrade approach or side-by-side approach.

Does the SQL environment also fall into the category of upgrade or side by side

Yes! It absolutely does. But as this requirement is tied to our sccm decision making they all play a part altogether. The series itself yes is more for the design upgrades for SCCM in general for its topology and architecture changes, and SQL Server plays a huge part in it.

If serious design changes needed to the done for SCCM and SQL, then a strong case could be made to proceed towards a side-by-side migration. Of course in-place upgrades can be done but whilst it has the convenience of keeping everything as it is, it does come with many risks which can be reduced from careful planning and DR for each part.

How does the SQL Environment play a part

Firstly its about assessing the current SQL server environment in which you have and that comes down to many points such as

SQL Server Version
SQL Server Architecture for DR
Backup Strategy

More information specific to SQL Server requirements which may be referenced on this article can all be found here

SQL Server Version

Starting from the top lets look at the versions which we have. Looking at the official documentation from Microsoft ideally we want to be on SQL Server 2017 if we want to be able to support scalability for the future.

I would say the minimum requirement version which you should be on ideally would be for SQL Server 2016 and then recommended cases I would say SQL Server 2019. I have come across a lot of organisations which still do utilize a SQL Server 2012 environment and whilst you could still use this, you are running on the brink of being obsolete as well as this version being deprecated further into the new release of the SCCM CB versions when it gets to 2107.

SQL Server Architecture for DR (Disaster Recovery)

Next layer is now how we have it designed and its readiness for DR. Common options range from SQL Server AlwaysOn, SQL clustering and even down to legacy options such as the SQL database mirroring.

But the decisions and how you should have this setup can vary depending on the kind of role which SCCM plays in your overall environment. For example some organisations may not consider SCCM to being a business critical service being provided, there a single SQL Server instance is all they feel is required and in some cases even consolidated all on the same server.

Of course we have to understand to if or when that point does change then the options range increases as you would then need to take into account the following things;

Process of Migrating SQL Server from a consolidated Box
Removing Single Point of Failure
High Availability for the Database

Now for environments who have this setup already they are a step ahead in terms of overall checks and prerequisites for pushing forward. But there is still the case of actually performing a DR test or analysing the structure they have to ensure that not only high availability is supported but how efficient it is during a disaster situation.

SQL Backup Strategy

Quite an interesting topic here, as one of the most common things we see here can be what I like to call backup overlapping .

Not only do we have the options in SQL to backup our databases and logs, we also have the native backup options within SCCM which can take a backup of the database and site via a site maintenance task. This can sometimes cause a bit of an overlap because of how potentially BIG your SCCM environment is. If you have both configured and the database is typically 50GB for example, depending on your schedule and storage requirements you are essentially saving 100GB of duplicate backups per each run.

Before you proceed any further its best to sort this out asap because this is a creeper but it can start to get more annoying until it takes on the head of being extremely problematic.There is more to this section which we can cover in a separate article in other methods and more explicit areas of backup strategies too.

Reporting Services

If this is something which may not be utilised, then adding this into the fold isn't a huge deal.

But if you do perhaps use an older version of SSRS and have custom reports within this, then indeed you will need to pay particular attention to the sections mentioned above in regards to the key factors.

Upgrading SSRS can be simple but it can break very easily and cause the reporting services role to have to be reinstalled again, thus possibly losing any custom built reports and any linked report configurations which you may hold. Its best to make a backup of ALL of these kind of reports before you proceed with any change altering actions.

Data Warehouse Implementation

A new feature which has been available for SCCM for a little while which has not been used as much. Having the ability to have historical information within SCCM when it comes to deployments and various client health and activity information is extremely handy, but there are also a couple of things to consider in this decision also.

Firstly you are given the option as to which tables of information you want to capture historical data for and its retention period. These decisions will of course impact the requirements of your SQL Server storage requirements as having too much historical data can cause the same issues as stated above and taking into account that you will now have an additional database to which may need to be highly available if considered a business critical service.

Next on Part 5

The next part we will focus specifically around the high availability plans. We did touch in this a little bit for the SQL Server requirements but we will look to expand on this area much more.

Microsoft Endpoint Manager: In-Place Upgrade or Side by Side Part 3 - Current Branch Upgrade Approaches

dwalsham@walshamsolutions.com (D Walsham) — Fri, 19 Nov 2021 16:02:17 GMT

Impact of Current Branch Upgrades

Introduction

With this part we will be continuing looking at the comparisons of our approach when it comes down to the current branch version upgrades. There can be still very surprising events and challenges which can still indeed arise from this though has become more robust and more simplified over time. But it can have a slight unpredictability when it comes to different environments hence we will look and outline the areas of interest which will help you decide as to which route you may wish to take.

What could possibly go wrong??

Famous words amongst quite a few!

Performing a current branch update is fairly simple, but we have to take into account the overall architecture that we have for SCCM because whilst the upgrade itself is fairly simple, the changes itself won't be able to be rolled back so potentially there could be some very big leaps or changes which will have an instantaneous knock on affect to the rest of your environment, which is definitely something which you should bear in mind when proceeding to go ahead and apply these upgrades.

Isolate device compatibility

This is probably one of the most sensitive parts of any upgrade, the overall device estate.

Now in real world scenarios you are always (well most) will contain a good portion of devices which may fit the following categories;

Windows 10 (Lower Recommended Build Versions)
Windows 8 & Lower
Windows Server 2008 R2 & Lower

These are just some examples of the overall mixture of platforms one would or could have in their estate. Now once you start to climb up in the versions of SCCM, especially where the SCCM client version is concerned you may start to see more and more additional resistance when it comes to advertisement of deployments, policy changes and software update patching and so forth. Eventually you may get to a part where advertisement hits and misses get to a point where they start to no longer function correctly. So much to the point that you would need to reinstall the client, sometimes even locally on the machine in order to force a refresh of these.

This is an issue regardless of slow or quick you might be to upgrade your infrastructure. And by the time you potentially reach up to SCCM 2006 , this is where the compatibility is officially cut. As server OS clients such as Windows Server 2008 R2 and lower are no longer supported. Resistance on client OSes such as Windows 10 1903 could also become a problem further down the line which we will cover later on.

More can be seen on the supported OS clients and server systems here .

Workarounds can be done to maybe keep somewhat of better communications such as having those unsupported clients at lower SCCM versions where they may have been once more responsive, but overall it can me more of a stalling or prolonging the overall situation as much more administrative effort is then required to ensure that they are communicating with SCCM.

Frozen Stages of Upgrade

Not sure how common this particular situation is still but for a long while this had caused a lot of issues.

Basically whilst running through the upgrade where you get the window stating its progress of how the upgrade is going, either through the prerequisite check or the installing check once it approaches the steps to do with checking the SCCM database, there is a tendency to actually freeze and not move anywhere. Even the log files which monitor the progress of this also don't move which makes understanding why it does this to be very complicated.

Tools such as CMUpdateReset.exe tool does help to get out of this situation, but I have also seen many cases where this doesn't work and processes remain locked and will not break. Worse case scenarios have came into play where sites had simply had to be rebuilt because of this.

Why is this a particular reason to consider an in-place upgrade or side by side you might ask? Well it can come into play when some of the issues related to this can come from a clogged SCCM environment which is then reflected within the database where peak times can fight for processes and having a not so updated SQL infrastructure doesn't help this situation either. So its something to bear in mind as well to observe other peoples potential issues which was quite common at a certain time.

ADK Version Hike

Similar to the section of isolating devices, as depending on the type of infrastructure that you have a huge jump in ADK versions can possibly be problematic. For example, certain ADK versions will become obsolete when it comes to climbing versions of the current branch build numbers such as versions for Windows 10 ADK 1903 being no longer in use by the time it starts to reach to SCCM 2103 and onwards.

A further breakdown can be seen here

Re-Evaluation of Hardware/Software Requirements

Perhaps this should be at the top, but then again with all the other categories collated together perhaps it makes sense to keep this here. Now when it comes to evaluation of hardware/software requirements this can be down from the OS versions which your mandatory site system server roles maybe supporting as well as other topologies such as utilising a CAS server, centralised storage etc..

Once starting to go up further in versions the more you will be faced with a decision to upgrade the OS levels to the recommended version for supporting the latest upgrade of SCCM which depending on your circumstances could potentially bring everything to a stand still.

And its not just those areas you also have to take into consideration the SQL Server versions as well. Anything 2012 and below will require an upgrade for sure and that can already have its complications of its own.

More information on supported software can be found here .

Take Advantage of the opportunity whilst you still can

A collation of various areas to consider, its best to take advantage of the situation before these types of decisions are approached before hand.

Further down the line there will be more updates and upgrades, and bottom line you want to ensure that you not only stay compliant but also ahead of the game where necessary.

A window of opportunity is really right now. Ultimately analysing and reviewing your current infrastructure on a HLD and LLD perspective will allow you to understand the required changes in order to move forward. Though this has been done ideally you don't want to just have to apply these changes right as you are faced with some of the bits mentioned above, it's best overall to have this settled before hand whether you select to perform this as an in-place upgrade or a side by side migration.

It's not about just being fully updated each time a release is thrown to you, but more about being resilient to changes where your estate doesn't get hindered by non-support, incompatibility and many other scenarios. All about a balance of being able to support scalability and being able to accommodate the new features and OS support levels as well as having wiggle room to support an estate that might not be quite the recommended level but at least a fair distance from being obsolete too :)

Next on part 4

The next part will have more of a pure focus on the SQL environment side which will be more technical as opposed to this particular part should also be comprehensive image wise too :)

Microsoft Endpoint Manager: In-Place Upgrade or Side by Side - Part 2 - Operating System Considerations

dwalsham@walshamsolutions.com (D Walsham) — Fri, 29 Oct 2021 14:15:14 GMT

Approach on Operating System Requirements

Introduction

From a quick recap from the previous part we spoke a little bit on some of the fine points between whether we would decide to go down the path of either an In-place upgrade or to perform a side-by-side migration.

In this particular part we are going to cover more about one of the big deciders and prerequisites in most conversations of this topic is the Operating System levels. These can influence our decision not necessarily dictate, but this really depends on the kind of environment in which you are dealing with as no decision will just be a no brainer or clear cut path for you to take without considering everything at hand.

Appropriate OS Level Scenarios

If wanting to see more detailed information on operating system requirements you can click here .

In a Perfect World

The smoothest way of going forward is always going to be to be on the latest operating system. It makes life so much easier especially when we take into account a decision like this. So if we look currently the latest server operating system we have available is currently Windows Server 2019 until Windows Server 2022 comes into play in more regular usage.

When we look at many system centre products most of the prerequisites regarding this requires us to be on at least Windows Server 2016 but recommended will be at the Windows Server 2019 level. Because realistically once newer versions come the current OS which is perhaps more popularly used will be obsolete quicker and therefore we end up running into this exact same situation in which we are currently talking about this blog.

And with this perfect world scenario it maybe "easier" to decide to go down the route of an in-place upgrade when it comes to the latest build version but it might not necessarily be an open shut case. See there is still the consideration of other architectural woes which may exist with one being if your environment is perhaps still attached to a CAS, lacks disaster recovery methods leaving it exposed for a single point of failure and perhaps even sites which could be retired. And also not to mention when it comes to areas of interest around your SQL infrastructure.

Those are still factors which could point you to the direction of going to a side-by-side migration route in order to preserve a working environment thus a smooth transition to a new environment if needed, but at least having the correct OS levels does help when making these decisions. Not to mention when it comes to the Windows ADK side when wanting to entertain OS management side of things.

The Real World

Now a trip back to reality....how things may really be as of now. As much as it is convenient as well as extremely handy on being on the latest OS levels, unfortunately this isn't really the case.

As widely used as Windows Server 2012 is, compared to the Microsoft Endpoint Configuration Manager versions mentioned it is considered very obsolete (support is still valid till Oct 10 2023 ). And of course there are environments which may even be more on the older side going back as far as....well lets just stick with Windows Server 2012 for now :)

One of the biggest hurdles you would have to face is not only just the performing of the upgrade, you've also got to consider the many hops in between to get up to the recommended level. And already performing one hop to a newer OS is already risky already let alone if you have to take more than one hop!

Which is why its normally recommended as a best practice to simply not risk your current working environment and just build a brand spanking new one in a side by side migration. However.....there may still be some reasons as to why you are still keen to do the in-place upgrades. Some servers can hold very business critical services whether that comes in form bespoke applications, SDKs, written code and many other forms, the servers can put themselves in a position where they cannot be removed. Some are even embedded in with many security policies whether that be old AD accounts, +COM permissions etc and these things may not be able to be replicated onto another server.

Now you might think well doesn't that support a scenario of moving to a side by side migration decision? Not really. Because you might have more of a chance of holding onto those critical servers with an OS upgrade rather than trying to move something which you may not know how to put back together if there is either no supporting documentation or any knowledge transfer.

Some of these reasons do go beyond the scope of a technical structure of this blog but every organisation is different and these things can and have come up and is definitely factors to consider.

Disaster Recovery Side

Upgrading an operating system is a risky process whether you have to do it or not, so its always best to take precautions.

Snapshot of the nominated server - If using a virtual environment a snapshot of the server is definitely a good way to start. Of course depending on the type of role that the server has you may want to consider snapshotting others if there are authoritative syncs that may send certain services of sync.
Backup of the SQL Databases - It can never hurt to take additional precautions regardless of a snapshot so if a server you are upgrading contains direct use of a database you will definitely want to back this up too.

What shouldn't drive your decision

What shouldn't drive your decision is a lack of. Meaning if you have a lack of resources being around disaster recoveries, no use of a test environment to build a POC with. These are reasons that SHOULDNT decide which route to go as they can cause grave mistakes. There are people who have a culture of daredevil to go ahead with an upgrade path without these things but even there is still careful planning in consideration.

But you should either fix those "lack ofs" mentioned above or at least consider the next steps carefully before adopting a we have no choice thought process.

Next Part

The next part will cover the current branch upgrade sides to see the impact of what happens we perform an upgrade and also look at the finer points of interest around its requirement levels and will be technical in preparation for covering each scenario.

Microsoft Endpoint Manager: In-Place Upgrade or Side by Side

dwalsham@walshamsolutions.com (D Walsham) — Thu, 07 Oct 2021 14:00:52 GMT

Part 1 - Analysing Scenarios for best method

Introduction

Here we are with another series. In this one we will be going through an interesting one which I think has also been covered many times in terms of step by step guides and some breakdowns of this particular subject being whether to upgrade your SCCM environment directly or to perform a side by side migration.

There will be many parts to this which we will look to carry end to end on not necessarily a how to do step by step guide perspective but more on a high level design approach.

General Consensus on Way to Go

It's always been said (well mostly) that the best option is to always perform a side by side migration which would be to build a new infrastructure and recover your SCCM environment to the new one as it provides the most security by maintaining your original environment whilst you can not only prepare and environment on the side but perhaps have a window of opportunity to add in some necessary tweaks and updates to it as well.

In perfect world scenarios I think this would reign true many times. However realistically this isn't always the case. First we should outline what are the scenarios that would prompt someone with making this decision. Lets have a look at a bullet point breakdown of some of them below;

Operating System : Requirement of high level OS i.e. Windows Server 2016/2019/2022
Current Branch Version : Wanting to perform a safer upgrade or may not even be on a current branch version
SQL Environment : May not be using SQL AlwaysOn or a sufficient SQL backup strategy for your environment
High Availability Plans : Requirements to avoid single point of failure
Hierarchy Downsizing : Wanting to retire certain sites or break away from the CAS structure
Environment Consolidation : Wanting to merge multiple SCCM/legacy environments/third party environments into one.

There are many more scenarios of course which carry weight to the decision making but these are perhaps the most common and most relatable to a lot of architects and system centre specialists.

With these scenarios being monsters its understandable why you would opt for a side by side option. The worst thing you would want to do is to be responsible for the only environment or the most business critical service to be interrupted.

Disaster recovery can also play a factor in this. Due to the fact there may not even be one or one which is not tested? You wouldn't want to wait for an actual disaster scenario to occur to know if it actually works or not :) This provides you with the opportunity to take action once you are in a position to start working on the new environment.

Why Consider In-Place Upgrade Scenarios

It can be a case of perhaps not having the luxury of resources to entertain a new environment to migrate to, but also can be down to how embedded those business critical resources are. We know that all environments are not the same though there maybe some similarities.

I have seen it where things such as security reasons can play a factor. Where depending on how the environment was built there could be some due diligence that may have not been carried out were certain security levels are unknown on deeper levels such as service accounts, ports, storage and many others. You can even come across environments which also may or may not have an HLD or LLD with them, hence where if you did go to a side by side migration then potentially you are looking at an entire redesign of the SCCM environment. Now that's not a bad thing at all, but a lot of environments can have caveats of business and time critical matters which make the likeliness of those options not come to fruition.

Now using the exact same scenarios in which we listed above, we will provide a small line or so to state as to what the reasons could be to support this option going forward;

Operating System: Embedded security hardening which may not be able to be replicated onto another server therefore an in-place OS upgrade would preserve this environment
Current Branch Version: Environment is already in a position which satisfies the organisation with sufficient backup and disaster recovery scenarios.
SQL Environment: Performing an in-place SQL upgrade due to same reasonings or able to just migrate the SQL servers as opposed to the whole SCCM environment
High Availability Plans: Focusing on just the migrations of storage i.e. storage for content library to support passive site server capabilities.
Hierarchy Downsizing: Detachment from CAS on current environment or retirement of primary/secondary sites.
Environment Consolidation: Adding to the same standalone site or if bigger device management can merge to new sites.

It can indeed save time to perform all of this on the same environment. But one thing that does have to be factored in is of course the high level of risk in which it carries. See performing in-place upgrades isn't necessarily straight forward. There are ALOT of processes which have to go into each part of the staging and if one of those goes wrong you could potentially damage your environment as well as the particular server in which you are looking to upgrade.

These are things to definitely take into account when approaching this.

What will we gain from this series

The aim is to cover each scenario in full detail and not to provide a bias to one side over the other. Now if we have learned anything over various best practices and preferential ideas is that the perfect world scenarios won't always cover a real world scenario so therefore we have to accommodate both outlooks. It's not about making or forcing someone to adhere to a specific narrative of this is the only way, but to understand what the risks and dependencies which are covered in each one.

We want to allow someone who is putting forth the design idea with enough evidence to backup the reasonings behind the decision and if that particular scenario is fitting. Comes down to the old saying of make the solution fit the business and not the business fit the solution.

Coming up on the next part

The second part we will be looking at the first bullet-point which we had mentioned in both scenarios being the Operating System. This is of course the foundation of anything moving forward before we begin to piece together the other parts as depending on which versions we might be on the OS is going to determine what we can and can't do in the long run. So the second part will analyse these steps and again accommodating both options.

Endpoint Manager: Device Estate Cleanup

dwalsham@walshamsolutions.com (D Walsham) — Wed, 06 Oct 2021 14:00:04 GMT

Upkeep of your devices within your Endpoint Manager estate

Introduction

When it comes to devices which you have enrolled into your Endpoint Manager, there can be a build up of either rogue or orphaned devices which can end up cluttering your statistics when it comes to looking into and extracting information to find out exactly what you have that is active.

This post will investigate and highlight some areas which can be useful to understand where we can help clean up these types of devices.

Though some of this information may seem obvious there is still relevance to understand what you actually have as manual tasks to clean-up these types of devices can really add onto the administrative effort which you already have. So hopefully we can identify these things and keep things tidy :)

All Devices Views & Reports

This is perhaps more of the manual side of looking at which devices are current by the last times that they have actually checked in.

Whilst helpful it may not always explain the overall reasons as to why devices haven't checked in. We have to take into account many scenarios such as devices could be perhaps turned off, not working, or even in some cases policy issues too.

The same results are provided when we look at reports. Here is a report based on Device Compliance below in Figure 1.2 detailing the compliance levels across all devices. Both options being the views and reports provide a more general information breakdown which would allow the administrator to see which are still active and not.

If connecting to the Intune DataWarehouse via PowerBI you will also get the same results.

Device Clean-up Rules

Quite similar options are seen in SCCM too where you have site maintenance tasks which can look through stale records. In this case we have a very similar option here with the device clean-up rules where we can automatically clean-up devices which are based on a number of days from the current check-in date as seen in Figure 1.3

You have a minimum of 30 days and maximum of 270 days to decide from. In my experience 30 days is normally considered orphaned, however there can be circumstances which can cause this. Lets say for example a load of devices experience issues with checking in due to issues with policies, authentication issues etc... now it may seem a stretch that someone could overlook and issue like this for 30 days..but it can happen :)

So caution should be taken when deciding this configuration. Very much so when it comes to SCCM too.

Intune Device Records/Azure Device Records Cleanup

This particular one here can be a quite overlooked bit. Especially for those who frequently provision devices when utilising Autopilot.

When removing or rebuilding devices, though you may have deleted it from places such as the Autopilot table and even the devices table, it doesn't necessarily get rid of the Azure AD device. Assuming that you are using Azure AD Domain joins in conjunction with your auto enrolment whether using Autopilot or not.

You might find that if you add a device back into Endpoint Manager in Autopilot, the device will automatically assign itself back to its original azure device record. Now if you are planning to use similar existing details this might be fine, but if not then this can lead to a mismatch on information and auditing when wanting to find out information on either record especially if different names are being used or naming convention policies are being applied in your Autopilot profile or simply added manually into the Autopilot device record.

You may even find resistance when trying to remove a device from Autopilot if the device already has an associated azure device record.

As you see in Figure 1.4 you have the Autopilot device record. If you click the Associated Azure AD device name, this will then take you to the azure device record as seen in Figure 1.5.

Once the record from Autopilot has been deleted the azure AD device would be eligible to be removed, but something to bare in mind when going through the process of reprovisioning.

Conclusion

Overall there are a few things to consider when wanting to get things tidy but also to show areas in which the clutter can indeed start to build up.

Modern Workplace Management: Creating Applications Common Practices - LOB vs Win32 Package

dwalsham@walshamsolutions.com (D Walsham) — Fri, 03 Sep 2021 14:03:30 GMT

Drilling into why we follow certain practices

Introduction

This will appear as quite a "we already know" kind of article, but there is a purpose for this as well :)

Everybody may not be aware, savvy or know the full picture behind why we do certain practices, follow a best practice or any kind of practice at that! So this article is really to outline specifically the process in which we use to create applications within Endpoint Manager, especially around Windows applications.

There are many different types of applications we can create in Endpoint Manager and it's important to highlight as to why we go with certain options when we make our selection.

The topic itself has been discussed numerous times before and may overlap somewhat, but what I want to achieve with this is to centralise a lot of the reasonings, practices and understanding and for one which may grow overtime as well depending on new changes or new updates which may come along in the future.

Line of Business Applications

So to summarise quickly, Line of Business Applications (LOB) are primarily applications which are packaged within Intune which are using MSI file extensions. They are used because they bring the most convenience in terms of just a quick upload of the file and pre-filling a lot of the windows which you would normally have to do if you were to construct a different application type such as a Win32 application or a Microsoft office 365 Application.

With its instantaneous setup you also have the command-line arguments preconfigured on the assumption that you will of course go with a silent installation and one which will not suddenly reboot the machine as you would prefer to control this from other areas within Endpoint Manager.

Below in Figure 1.1 is the example of a Line of Business Application configured in Endpoint Manager.

Win32 Applications

Win32 Applications are a very similar process which you would see in the Applications process within SCCM when packaging an application which doesn't use file extensions which pre-fill majority of the fields such as MSI, App-V etc

Instead of having an easier configuration with the LOB applications you have a much wider range of options to configure the overall package and also deployment of the application, options in which we will cover a lot closer in the sections below.

Win32 applications take on a .intunewin extension which have the requirement of source files being converted into a .intunewin file extension before it can be uploaded and configured into an application. Below in Figure 1.2 is an example of a Win32 application configured within Endpoint Manager.

Why we choose Win32 Applications over LOB Applications

The title does seem bias, and it is :)
There has been a lot of real case scenarios as to why we go with one over the other but isn't without good reasons. And some if not all of the categories below I'm pretty sure a lot of us are familiar with. But what we want to do is perhaps dive a little bit more in the perspectives of other roles which people might have as reasonings can always differ depending on the situation or scenario at hand.

More Options for Deployment

Though the benefit of convenience gets robbed from us when we go through this type, there are a lot of benefits which give the administrator a lot more authoritative control over the whole packaging process. And that is very important taking into account the requirements you have.

Here are the following categories which we get when going through this option;

Program (Install/Uninstall Commands, Return Codes)
Requirements (OS level and system requirements)
Detection Rules (How the application is detected as Installed or Installed)

As much as we would love to have every package as an MSI (might not be a shared opinion) in a real-world scenario not every application is going to be configured that easy, in fact MSI packages also would be subject to the same drawbacks which we will also cover later.

Program

Here we define what the commands will be for Installs and Uninstalls, as well as its behaviour for installs and restarting.

Now in comparison to an LOB application installs and uninstalls are configured as well as the assumption of suppressing a reboot and notifications. But what if we require a reboot to take place? And if we do how are we able to have better control on its behaviour?

The return codes can also be an important factor as various applications may not be built to end with a return code or exit code of 0. And if we look at that from an MSI package perspective if it wasn't a 0 (depending on how its built) it could always be perceived as a failure. So being able to understand the return code output and having control over that is very important.

Requirements

This is also a very important section where we can define as to which applications will actually receive this deployment which would be determined by its OS and System Requirements. This does in fact bring us some level of convenience here. As with an LOB application we don't get this configuration, so we would have to carefully configure an equivalent by creating a group which would then have a similar configuration within its dynamic membership rules.

You can of course have a manual membership rule which would be a lot harder to maintain, but it would assure which devices are receiving the deployment and which should.

Detection Rules

Having more control of knowing how to detect whether an application has been installed/uninstalled is also a very handy option to have.

Again very similar to SCCM where we have options for using Registry, File/Folder detection methods as well as MSI and defined custom scripts. We will have more on detection methods in another article in the future.

The LOB Application using MSI simply just uses the MSI product code to detect its install, but depending on our requirements it may not be as simple as that unless its packaged within the MSI the correct way to check all of this, then we would need an alternative method which would be accurate for us.

Packager Perspective

This one is quite interesting especially if you are a packager and then uploading applications into Endpoint Manager for deployment.

When we package applications, there are so many things to take into account when it comes to the decision as to which option we would go with and the majority of the time it would tend to be Win32 Applications and here are the reasons why.

See the output when we normally package applications is going to be MSI. But of course as you would have seen from a lot of the options we get from a Win32 Application template, there can easily be a lot more bits to consider when it comes to the overall configuration of the application especially when further customization is involved. And if they are not already in an MSI then the additional configurations can derive from an MST file. Sometimes not even an MST file, but we could also have loose .CAB files which contribute to the MSI application being installed and configured correctly once deployed.

So lets step through for example the Programs section. Now with the install and uninstall commands, we can of course use an msiexec command, there's nothing that stops anyone from using an MSI package from being configured as a Win32 app, the reasonings and further experience configuration is what drives us to use this. Within a LOB application configuration you can upload an MSI yes, but you won't be able to upload the MST and the additional CAB files which are required for it though. You maybe able to do this if you compress an MSI file with the CAB files to be embedded but depending on the application you are packaging it could be come quite sizeable so that's something to be mindful of.

It's installation behaviour is also something to take into account as the application packaged may very well require a reboot to take place so this is where the behaviour experiences become a lot more important. And the even more important one of course is the return codes! We may have multiple return codes depending on how we have built the application so we want to reflect that so when it comes to collecting logs we can analyse further as to if something has been successful or not as well as the default mechanism within Intune.

Issues with mixing LOB & Win32 Apps

The general rule has always been not to mix the two of them in the same processes, specifically naming the Autopilot process.

Reason being? Is because it's a lot more prone to errors. I have seen a lot of cases where within Autopilot provisioning you will see applications literally hang or not move when it gets to a certain application. And though this is randomized if you have a mixture you could have a good guess as to the type of application which could be causing this.

There isn't really an easy way of getting around this so its best to just not mix the two to avoid complications.

Error Handling

When we troubleshoot failures from deploying applications in Intune, and MSI packages in general the most common return code you will most likely come across is 1603 which is a custom action error. This can cause a lot more issues when it comes to troubleshooting overall.

Logs within C:\ProgramData\Microsoft\IntuneManagementExtensions will only reveal so much which is really more around the error code itself. So really the more detailed logs will come from the msiexec switch being /l*v <LogFileName> , which could be placed in the same location as mentioned above which can also help when collecting logs from the Endpoint Manager buttons.

Doing this troubleshooting may be trickier in a LOB only because you might use the same switch and discover the issue but if its related to a missing CAB file related to its MSI tables, or an MST is missing it could lead to more repackaging or just longer amount of time to understand what is happening.

Conclusion

In closing there are a lot of things to consider when it comes to selecting any option when it comes to creating applications or migrating applications across to Endpoint Manager. Any decision made shouldn't go without its testing but also analysing the scenario in which you have and how to best configure the application correctly as well as it working on the endpoints in which it is distributed to.

Modern Workplace Management: Build Endpoints Without SCCM Part 6

Thu, 12 Aug 2021 14:11:19 GMT

Analysing Hybrid Scenarios

Introduction

All the parts of the series we went into great detail about how we analyse an end to end solution and how we would design a solution in which would allow us to build endpoints without SCCM being a dependency.

Whilst we did this, there is another scenario which we have not touched on yet, which is the hybrid scenarios.

In a perfect world ideally you would have your Azure Active Directory within the cloud, every machine meets the recommended requirements for Windows 10, everything is imported into Intune/Autopilot and everyone is happy.

But we know this isn't realistic in all cases. Many organisations cannot just simply up and go from on-premise into the cloud therefore the checkpoint here is of course getting into hybrid solutions such as;

Co-Management Between Intune and SCCM
Hybrid AD with Azure AD and On-Premise AD syncing together

These things can play a very interesting part in how you would tackle this if you envisage the next step in the blueprint is to be in a position in which you can build and manage endpoints soley within Intune.

With this final part of the series we will go in-depth in how the common hybrid setups look like and how we go about moving into the next step of being able to manage and build devices without SCCM.

Current Hybrid Design And Process

First we need to outline in high level how a typical hybrid solution would look, very similar to the 2nd part of this series when we looked at a typical setup in a perfect world scenario.

Once we have a view of this we will understand the role in which endpoints play once they go through the workflow cycle of being a device managed by Intune/Autopilot.

Below in Figure 1.1 we can see a detailed diagram of how a hybrid solution is currently looking like.

Looking at how this is laid out we will then also draw out how this would look for an endpoint going through this process as we see it;

A new device gets built via MEMCM Task Sequence
Device is then provisioned to be an Autopilot Device
Device goes through the Autopilot process once the user has logged in with their credentials
Device then is joined to the Azure AD and also the On-Premise AD
Endpoint is now ready managed by both Intune and MEMCM as well as Azure AD and On Premise AD

There are of course a few little bits to configure before it will just get you to the point of being in a hybrid AD environment which will go into further in the next sections below.

Notice though in the diagram I have left the Non Provisioned and Provisioned devices there. Reason being is because yes the main goal is to eradicate the categories, but as a lot of us maybe in a hybrid scenario these categories will still linger as we still have devices coming in or registered in multiple places such as;

Autopilot Only
Intune Only
SCCM Co-Management
Azure AD Joined Only
On-Premise AD Only

So the next sections will help to consolidate these paths where once we get devices from all areas centralized we can then start to work on the removal of the categories of non provisioned and provisioned.

Consideration of the Hybrid Connector

Of course one of the next things you want to do is also be able to get your devices enrolled into Autopilot. Given the hybrid scenarios and of course with the addition of an on-premise Active Directory you have another option. For where you have your on-premise computer accounts in AD, you can utilise an Intune Connector which will allow you to have the same computer accounts created within your on premise AD so that they are ready to go when needing to rebuild those endpoints again.

The process works by taking devices which have been already provisioned within Autopilot and then creates them within the On-Premise AD, so when it comes to the provisioning process via Autopilot the device is then ready to go once the endpoint has been enrolled.

If you have an environment where you have a multitude of devices in Intune/Autopilot which are not on-premise joined already then this is most certainly an option to look into.

An extensive guide is available for how implement this here

Co-Management Affinity - Between SCCM and Intune

For those which use Co-Management we are at that safe point where we cannot call as to which direction we want to stay in and that is most certainly fine. The overall solution is not to necessarily make a case where we dump SCCM as such, but to at least remove a core dependency of the responsibility of building devices.

Now in a Co-Management case, once we have a nominated device collection and Azure AD group in mind, those devices will then have records within Intune now and you will be able to identify them in the properties of the Intune devices by looking at it's Managed By section reading as Co-Managed.

Depending on the setup you have with devices which are solely manged in SCCM, there is a high chance your devices will primarily be managed within an On-Premise AD unless you are using the Azure AD Connector or have it where devices are using a work or school account configuration to be managed in Intune without an auto-enrolment in place.

But with Co-Management, this is primarily to get Windows 10 Devices managed which are of course eligible, but also means there are devices within your estate which you cannot manage by Intune therefore SCCM is still needed. The devices which are in Co-Management can be eligible for the Intune Connector to prep them for autopilot deployment as well as the whole provisioning process which will allow them to be Hybrid AD Joined afterwards.

Preparing Intune for Hybrid Autopilot Deployment

Taking all of the previous parts into account as well as the design prep considerations above, we just need to tweak a couple of things to ensure that everything is good for supporting the hybrid scenario.

Domain Join Profile

This configuration profile is primarily for your on-premise AD, and this will be applied during the autopilot profile is being deployed to your endpoint.

The domain join profile does have some prerequisites. One being the actual Intune Connector. The connector would have had permissions delegated to an OU in which it could be able to create the Autopilot devices with, this is also the same so the relevant delegated permissions need to be in place for this to work.

Once this has been completed ensure that you deploy the configuration profile to the group which has your autopilot devices so when it comes to a reset or rebuild this will come into play during the autopilot process.

Autopilot Deployment Profile

To ensure that our autopilot deployment profile also accommodates our hybrid setup, we need to edit this so that we are able to use a Hybrid Azure AD Joined profile.

Now you may not be able to edit a current one so you may have to create a new one which reflects the settings below in Figure 1.3 specifically around the Join to Azure AD as line

Third Party Option?

So for those who are moving towards a third party to provision devices before they are shipped either to the user or to the organisation itself to distribute them, the hybrid scenario is something which the third party will need to accommodate.

There maybe a reason to bring a VPN into the scenario so that devices can go through the hybrid join process so that when they arrive they will be ready to be supported. An always on VPN solution could be ideal for this scenario indeed.

End of Series

This now concludes the whole series. Hopefully this has covered a lot of areas well and looking to come back with some additions to this at some point soon.

Modern Workplace Management: Build Endpoints without SCCM Part 5

Thu, 29 Jul 2021 17:10:12 GMT

Backend Configuration within Endpoint Manager

Introduction

In continuation from the previous part where we had discussed how we create the "on site" piece of the solution, this was the part which would allow us to get our endpoints into a state in which they would essentially be ready to go through the Autopilot process.

Which leaves our next piece of the puzzle, to begin the configuration of the actual backend side that resides within our Endpoint Management console. And you will see how everything ties up together to satisfy the full end to end process of getting an unknown (or known) device to proceed thorough the whole workflow to be finally managed by Intune without the aid of SCCM taking part in any of the prerequisites or preparation at hand.

What do we need to Complete the backend configuration

So of course the obvious part of this is going to be that our auto-enrolment side is setup which is most certainly a given, but in this case we want go a little bit more granular in what we specifically need to get this going.

To get to this we need to perform the following;

Define Company Standard Applications
Define Company Standard Profiles

Create Autopilot Profile
Create Enrollment Status Page

Define Company Standard Applications

The aim in this exercise is not to replicate a task sequence or gold image as such, but really to define what are exactly the absolute minimum types of software which every standard user requires before they can be up and running.

Best way to define this is if we were to lets say make a list of every application which we envisage a device would need as a pure example;

Microsoft Office
Google Chrome
Winrar
Winzip
PowerBI Desktop
Amazon WorkSpaces

Just really a few to list off as other companies standards could be much more extensive than this. And that's exactly what we want to account for. See if we have lets say 30 applications that we require a device to have. Do you know how long that could take to deploy to every single machine which went through the Autopilot service? A very long time indeed!

So how do we overcome this? We define categories of priority for these applications. And we would do it like this;

Which applications are top priority?
Which applications don't have to be installed during Autopilot but still be mandatory?
Which applications can be made as available for users?

And the aim of that list is so we can cut down on the overall Autopilot process so that devices are not in a backlog and can make a turnaround of being completed a lot more efficiently.

To make another example of this lets combine the apps we listed with the category question list above;

Which applications are top priority ? Microsoft Office, Google Chrome
Which applications don't have to be installed during Autopilot bit still be mandatory ? PowerBI Desktop, Amazon WorkSpaces
Which applications can be made as available for users ? Winrar, Winzip

Now the example applications list I made isn't too extensive so I would probably be saving seconds rather than minutes :) But when this list does scale up it will help the planning for the future as well as right now if you are at that point.

Define Company Standard Profiles

This is of course a monster of a type of exercise but will give some information on this briefly to cover how you would define these.

Things you perhaps want to take into account is that this is structured on a migration from on premise to cloud type of transition, so if you have policies or settings which could be applied to endpoints solely on SCCM then you may want to bring these over so that you can maintain the integrity of company standard policies. The transition bits are covered on a previous series I did on how to migrate from SCCM to Intune which you can check out :)

Create Autopilot Profile

The Autopilot profile bring us even closer to complete the full workflow as this is where we will be able to allow endpoints to be enrolled into the Azure domain and automate (or semi automate) the configurations within our OOBE (out of box experience) once the user has successfully authenticated with their device after first login once the Autopilot Profile has been detected.

So going into the Devices - Windows - Windows Enrollment - Deployment Profiles we can go ahead and create our deployment profile. Below are the settings of an example one here;

Now this also has an important bit checked which is called Convert all targeted devices to Autopilot. This allows for any machine which is not registered in Autopilot, but perhaps is registered within Intune to be converted to an Autopilot device. Which means that if we were to reset that specific endpoint for whatever reason, the Autopilot profile would kick in to help build that device again.

( Note: Bear in mind depending on the structure of your environment this could be a risk depending on where this is applied. Ensure that there are NO applications and NO profiles being deployed to ANY group which is specifically for Autopilot devices which you may not want applied!)

Figure 1.2 shows the OOBE settings which is where we want to make the user experience of going through this device very simple. Ideally when going through the OOBE stage the user will be doing this on their own, unless you have a process in which a service desk team may perform this with them or for them.

A popular method to use can be the Allow White Glove OOBE configuration where you can have the device complete the entire Autopilot process when no specific user has been identified and to simply hand or send the device to a user of your choosing. This can only strengthen the overall design of building endpoints without SCCM.

Next part is then the assignment of this profile, here we have a group which contains only Autopilot Devices. Now you don't necessarily have to deploy it to a group like this where it may contain the dynamic membership rule which will pull in ANY device that's in Autopilot as seen in Figure 1.4. You can also have a static group where you can place devices in if you wish to stagger or semi automate the process before fully committing a group to this process.

Create Enrolment Status Page

In my opinion this is a very slept on part of any piece which involves any part of a windows enrolment process. So many of us stick to using the default All users and All devices page which may have no specific configuration within it.

This is super important to this series. Why? because if we look back at the exercise in which we did for defining company standard applications we had some applications which we said we COULD NOT do without and they were the absolute most critical to be on a device before it would actually move along. This, is where it's all controlled at!

So once again navigating to Devices - Windows - Windows Enrollment - Enrollment Status Page you should see the default page mentioned. Here we will create a new Enrollment Status Page specifically for our Autopilot process.

Once we hit Create and provide a name we should then get the following settings below. Very typical settings which simply allows for the device to display the progress windows which perform the domain joining, application installing and other forms of configuration. But the one setting we want to configure is the Block device use until these required apps are installed if they are assigned to the user/device seen in Figure 1.7

Here I click Selected and then choose the applications which I want to be installed first, being the ones which we had defined earlier as the absolute top priority ones. This way once those apps are completed the device is ready to go and doesn't have to be held up by an excess amount which wouldn't stop an endpoint from being defined as ready.

The last part is of course where we deploy it to. So we can only deploy it a group, so we select the group which has the Autopilot Profile assigned to it being the POC - Windows 10 AutoPilot Devices group.

With everything now configured on the backend, you should now be able to perform the following;

Reset a device which is in Intune but now converted to an Autopilot Device (Known Device)
Manually import a device into Autopilot to have its profile assigned (Unknown Device)

As both will go through the same process, we have now consolidated known and unknown, to just "known go ahead and reset" :)

Next Part

The next couple of parts on this series will be focused around how we tackle this on a hybrid environment where we have an on premise Active Directory and an Azure Active Directory. May find most of the previous parts have covered the majority but there are little things which can cause great hurdles in trying to achieve the same goal.

Modern Workplace Management: Build Endpoints without SCCM Part 4

Thu, 15 Jul 2021 10:49:27 GMT

Build Prerequisites Process within Endpoints

Introduction

In this part we are now going to look into the technical step by step points on how we put everything together. In the previous part we spoke about the structure of how we would asses whether a machine was actually ready to be built with Autopilot or not with a build checklist process which would step through all areas which would cover an endpoints eligibility.

Now with everything planned out we finally want to step into making things reality by putting everything together.

Automated Reset PC Process

We will look at how we construct this technically but we will also look at two ways in which we can do this. But the whole idea is to make this a script based process which will automate the assessment of the device before we say OK its in a position to be reset and then be put in a position to be put through Autopilot...or even whitegloved.

BIOS Checks Section

For the BIOS checks as we mentioned we have various methods of how we would check settings as well as set them such as;

Enabling Secure Boot
Enabling TPM
Ensuring Virtualization is Enabled

Now if we take a look at the Lenovo Manufacturer , we have WMI & PowerShell areas in which we can automate the checks of these areas. But we also have to take into account that we may have multiple vendors for our hardware in the estate, which is where we formulate a structure which is able to detect how we find out if it's a Lenovo based device.

So looking at the first section in Figure 1.1 we have a done a few things for this process;

Obtain the manufacturer details
Detect if the device is a Lenovo manufacturer
If it is then obtain all of the BIOS information details
Ask for BIOS Password

This first section focuses on the capturing of information currently configured on the device where the BIOS is concerned. Which covers everything we talked about in the bulletpoints above.

Once thing we need to take note of which is quite an important line which is the $mypassword variable.

When it comes to the Lenovo BIOS settings via WMI, one of the things you simply CANNOT do is set a BIOS password for a Lenovo device where no password was set beforehand. This can only be done if a password was already set beforehand, hence the reason for the line which asks for a BIOS password as in order to make any changes to the BIOS via this method you need the password in order to make any BIOS changes.

For more information on the Lenovo WMI BIOS settings you can obtain a guide here https://download.lenovo.com/pccbbs/mobiles_pdf/kbl-r_deploy_01.pdf

This section is also great for when you have multiple vendors where if you want to add additional vendors such as HP then you can make another section in your script to include this.

The second part of the BIOS checks is where we do the verification of the BIOS Password as stated above. There is a line in there which will essentially stop the script running any further if no BIOS Password has been set. This would allow a user to set the BIOS password manually on the device and they can kick the script off again when ready.

The TPM verification will check if its enabled, as this is quite important when wanting to use Bitlocker as well as managing its recovery keys. If the TPM chip is not enabled then it would enable this feature.

Final part of this section looks at the verification of the Secure Boot and Virtualization Technology.

The Secure Boot feature has the same caveat as the BIOS password where it cannot be enabled through this, so I've placed some lines which will say if it is enabled or not. If you require this enabling then you would need to do this manually on the device.

The virtualization setting on the other hand can be enabled if detected its not enabled already.

Device Autopilot Provisioning

This section is really optional as in the next part we will show how to get all of these into Autopilot anyway. But if you do have a policy in which you want to stagger which devices would get into Autopilot then this section is perfect.

So this will do the following;

Obtain the serial number of the device
Prompt you to connect to your Intune tenancy
Check if the device is in Autopilot
If not then it will add it to Autopilot
Then will check to see an Autopilot Profile has been deployed

Device Record Checks

Again this could be another optional section if you are using SCCM to a certain level.

If using Co-Management or just using SCCM in a normal fashion, if the endpoint you are working on also contains an SCCM agent then there could be a potential issue where you could have duplicate records once this machine is rebuilt This could even happen when having an Active Directory System Discovery which would discover an obsolete record there.

This section will check to see if the record exists in SCCM, if it does then it will delete it.

Reset PC

Once everything has gone through and been validated up to a standard which we are happy with, then the last part is to run the systemreset command which will prepare the machine to go back into an OOBE (Out of box experience) state

How to deploy the Automated Reset PC Process

Locally on Machines

When we think about how the devices are managed on the floor we think of service desk and hardware support. In cases where we want to kick the machines off to be reset we would kick the script off on the machine.

The best way to achieve this would be to create a Win32 Application which would be deployed to all of the devices which are also registered within Autopilot so when it is kicked off, we know the machine will already have an Autopilot profile already associated with it.

Proactive Remediation Script

Another way is to perhaps split these sections or not and have them run as a Proactive Remediation script.

The aim would be here is to be able to have a reported amount of how we would deem "eligible endpoints" that can indeed be reset. Whilst the scenario above is great, we have to take into account the remote aspect of things.

So for example say we create a brand new group in Intune, and then we build a dynamic query of devices that are all based on a Lenovo Manufacturer.

Then we take a new template which we have which is based off the Automated Reset PC script but we split this up so we will focus on this being the Lenovo BIOS settings section.

So we then deploy this to the Lenovo Devices group, and this would then check to see if all of our BIOS Settings are enabled and this would mean they show as compliant. If not then they are not compliant.

But the important thing is that we would have a list of which machines are deemed as ready to go. And then we can go ahead and hit the Wipe Device button.

Of course depending on your setup either a service desk user would be able to control the other side, or instructions for a user which would allow them to just log right into the device using their email or login credentials which would set the device up right after it enters the OOBE state.

Next Part

The next part we will focus on how we do the technical steps on the Endpoint Manager side so that we consolidate the overall workflow between the Automated Reset PC process and the back end of Intune.

Downloads

You can download the template scripts used in the article right here to reference from.

Endpoint Manager: Find machines not in Autopilot

Fri, 02 Jul 2021 12:36:51 GMT

Find the amount referred to in your Endpoint Analytics Baseline

Introduction

When it comes to managing your endpoints in endpoint manager, one of the things you may be looking to do is to get all of your Intune registered machines to also be enrolled as Autopilot devices.

Now we can of course just have the deployment profile deployed to all machines and then hit the "Convert targeted machines to autopilot" but this might not necessarily be feasible for every client. We may want to perform some due diligence first so we can at least understand what devices in Intune are not in Autopilot.

Baseline In Endpoint Analytics

When you go into Reports - Endpoint Analytics you should see the overview of our whole estate with the Endpoint Analytics score. But if you also see on the far right where the baseline toggles are is the Insights and recommendations which indicates how many by percentage are not registered with Autopilot.

Whilst this is good information to know, there isn't a direct way of seeing exactly which machines are actually not registered, and this is where we come in.

How to Discover which machines are not in Autopilot

So the first thing we want to do is to be able to connect to our Intune tenancy which we can use the following

Install-Module AzureAD
Install-Module WindowsAutopilotIntune
Install-Module Microsoft.Graph.Intune
Connect-MSGraph

This portion will allow you to connect to your Endpoint Manager tenancy using MS Graph. As we are not pre-caching any credentials you'll need to login manually.

You should then see the connection details of your UPN and TenantId. Once done then use the following commands

$DevicesInIntune = Get-IntuneManagedDevice -Filter "operatingsystem eq 'Windows'" | Get-MSGraphAllPages | Select Userdisplayname, devicename, serialnumber, manageddeviceid
$DevicesInIntune | ForEach-Object {$_.serialnumber} {If (Get-AutopilotDevice -Serial $_.serialnumber)
{$_.Devicename + " " + $_.serialnumber + " Yes"} else {$_.Devicename + " " + $_.serialnumber + " No"}} | Out-File C:\Temp\Autopilotnotregistered.txt

This should then produce a text file with all of your machines according to what you have in Intune so that you have an accurate understanding of what is actually in Autopilot and what isn't.

Tips for Intune Device Filtering

Bear in mind that when you run the Get-IntuneManagedDevice command with no filter you will get every single device, whether it be Windows, iOS or Android. This is important to note as you may get every single device and it won't make much sense as Autopilot is only for Windows devices, which is why the code above limits it to only Windows devices.

You can however change these filters if you wanted to limit it down to a specific vendor.

For example Get-IntuneManagedDevice -Filter "Manufacturer eq 'HP'" if requiring to narrow the search down where required.

Modern Workplace Management - Build Endpoints without SCCM Part 3

Thu, 24 Jun 2021 16:31:28 GMT

Consolidation of Build Checklists

Introduction

So just to recap on the previous part of this series we covered a base scenario of where we would utilise other products as a dependency to provision or even pre-provision endpoints before getting them to a state where they can be auto-enrolled and then into Autopilot - in this case this would be around SCCM.

We also detailed in a diagram form of where we ideally want to be, and that's to not have any dependency when it comes to the provisioning of the endpoints where its also soley performed within Intune.

This part we will be going into the nitty gritty of how this works.

The "Provisioning Phase"

This is the part which was not visible in the diagrams. Reason being is because we want this to be a seamless transition, embedded part of the process that gets you from point a to point b.

But first, lets explore how we see the provisioning phase working below.

This diagram gives a flow chart which mimics how we would see a typical build checklist would go, and then see it go through the motions where we would then have the machine prepared within this process and then we can say yes its ready to go through Autopilot or even the white glove process.

In real world cases this process is split up where prerequisite checks can be of a manual process, where as there are scripts which can easily add a machine into Intune & Autopilot, more notably the Get-WindowsAutopilotInfo.ps1 script

Manual Checklist

Depending on manufacturer these can range from various little tasks such as;

Ensuring TPM is enabled and using 2.0 for using Bit locker
Secure Boot if using a UEFI boot using something like SCCM
Time & Date Settings in BIOS
Standardised Boot Order
BIOS Updates (optional but can vary)
Removal of existing device records (i.e Active Directory, SCCM, Intune, Azure AD)
Checking if device exists in Autopilot already

Now if you put that into practice of an extensive list like this and having to go through several thousands of devices the administrative effort can indeed be overwhelming. Now for new devices which are prepared from factory and already added into Autopilot then this list isn't necessarily an issue.

Having said that, that doesn't mean that any of these settings haven't changed in anyway so essentially an engineer would still have to double check just for the case of due dilligence.

Automatic Checklist (Task Sequence)

These are the somewhat obvious bits when looking to get machines enrolled in masses. A task sequence that would contain things like;

Enroll into Intune
Add device to Autopilot
Check if Autopilot profile has been added

Of course these can be done manually, but you don't want to be doing this manually :)

Tasks such as these are consolidated within some form of a Task Sequence to be able to deploy to mass amounts of devices. But as you can see with both the manual checklist and automatic one we have a semi-automatic process altogether. There isn't anything wrong with this as semi-automatic approaches have a benefit of at least being checked over with some form of human interaction. But we would like to get to a point we can automatic and logically structure this process as close as possible.

How to Consolidate this phase and not using SCCM

To reiterate this section is applicable for management of all existing machines within your estate and not for machines which are brand new. These ideally would already be provisioned and ready to go. Having said this a new device will turn into an existing device so bear this in mind.

Reset PC/Wipe Device

Very simple and straight forward option to go with when needing to reset the device to go through the process again if giving the device to another user.

Of course this option is useful if you are absolutely certain that everything has had a check so that it had all of the standardised requirements, and that's more on the hardware front. Depending on the kind of setup you have whether it would be an Azure AD or even hybrid scenarios (will be covered in a later part) there are still other technologies you would have to consider to perform clean ups on before you can move forward.

Take for example Active Directory and SCCM/MEMCM. If you are going through the Autopilot and joining the machine to the domain whether the scenario you could run into issues where the record already exists and therefore has an issue joining it to the domain and that would have a knock on affect on the Autopilot process.

Then looking at the SCCM/MEMCM side of things, this isn't necessarily as critical but you could face duplicate device records and if deploying anything via SCCM still can also cause issues too.

The wipe device method is also great. Bear in mind your device would have to already be managed by Intune at this point, and is seen more of a security measured in case a device is lost. But it does have its benefits when wanting to reset a machine too.

Automated Reset PC Process

Here is where we want to add the magic.

What we ideally want to do is to turn those manual checks into a process which goes through these seamlessly and then prepare the machine to be reset. Best thing to compare this to would be similar to how you would have seen a Windows Feature update be applied.

First thing it does is check all of the prerequisites, mandatory ones of course if that fails then the upgrade will essentially stop but if any optional ones flag up it wouldn't halt the process. And this is what we want.

Figure 1.2 - Shows how we want it to be as just one process, and not split up into multiple tasks. But lets perhaps cover the tasks in which we have mentioned just to give a high level overview of how this can be automated into a process.

BIOS Checks
So we mentioned earlier about things such as

Setting Date & Time
Enabling Secure Boot
Enabling TPM
Ensuring Virtualization is Enabled
Organising of Boot Order

Now the key part of any of the categories is not so much whether we can change them if they are not set correctly (we are aiming to do this too though!) but more of how this is communicated to you. To think of how the administrative effort is lowered by a process telling you rather than you having to manually check to see if that's actually the case or not.

So things like WMI classes can detect and set a lot of these. If we investigate the Lenovo manufacturer for example, you can use WMI classes to not only detect all the ones stated above but also to enable/disable them at leisure. More can be found about this here

As an example of what we mean of being told what the deal is with an endpoint or not, say we use BIOS passwords to lock down settings on endpoints. Now for the Lenovo WMI Class you cannot set a password for them if it was never set before . You can only clear it or change it. So with that scenario we would engineer it where we are advised of what the password state is. And if it is set then we validate that it is using the correct password.

Now in a regular way of performing this you would have to shut the machine down, restart it and go into the BIOS to see if this is the case.

Removing Device Records
This can be tricky. Perhaps more on an Active Directory front.

In an ideal world we would love to automate the removal of device records from technologies such as SCCM, AD etc but there are some factors you have to think about, especially since I've experienced this.

Say for example you remove the computer from AD but you either accidently or purposely reboot the machine for whatever reason, you can't get back into the machine after that. Unless you have a local account on the machine you would have problems. Now you can wipe the device from Intune ONLY if its managed in Intune and you can ONLY Reset This PC if you haven't used anything such as Bitlocker. Why? because it will ask you for a recovery ID and if you don't have that at hand you will simply have a dead machine.

So we want to have a process for a section such as this to do the following;

Detect whether the records exist or not

Detect whether a local account exists or not, if not then create it

Proceed to remove account in preparation

These are example steps of the logic, but we want to ensure that all basis are covered because these are key areas where automating could go wrong.

Enrolling to Intune/Autopilot

This part is mostly in best practice done within an SCCM task sequence. And what we want to do is take SCCM out of the equation for this process.

So we utilise the scripts that perform this step, extract the commands out of it and then build the automated reset pc process with it which would do the following;

Obtain the device serial number
Connect to Intune
Produce the hardware hash
Check that the device is already in Autopilot
If not then upload it to the Autopilot
Check to see if the Autopilot Profile has been deployed (optional)

So the idea behind this logic is to really see what is happening so that we understand the devices current state in the enrolment side of things, and which is also the last part of the process to tie in altogether.

Next Part

Next part we will show a template which can be followed to automate this process for when it comes to re-provisioning devices and also cover the process parts in the Intune/Autopilot side. Will be quite a lengthy part so will be coming soon as!

Modern Workplace Management - Build Endpoints without SCCM - Part 2

Fri, 11 Jun 2021 17:27:47 GMT

High Level Overview of the Current & New Build Structure

Introduction

So on the first part of this series we covered the synopsis of what we are really trying to achieve which is to basically design a process which enables us to not have any dependencies on any other technologies when it comes to provisioning endpoints and managing them within Intune, in this case we are talking about SCCM/MEMCM.

SCCM indeed will always play a big part and for those organisations that still use it and is a huge footprint in the overall design it's not a simple process to completely leave SCCM to manage everything solely in Intune. But what we can do is start by lessening the load of what SCCM is responsible for and try to allow Intune to be more of the lead when it comes to endpoint management, which is really the core of when it comes to Modern Workplace Management.

How we currently Provision Devices

This may not necessarily be true in every case, but for where most of us in a real life scenario in which we utilize both SCCM and Intune, it will most likely be an option to use SCCM as a dependency and a step in between a device which is not provisioned to being auto-enrolled and provisioned.

These tasks can of course be done manually. But when you consider the administrative effort of how long it would take to provision thousands, tens of thousands and perhaps even more than that! Then this will definitely be a hurdle of course.

So if we look at the design below this gives a depiction of what we may most likely see our setup being.

So if we look closely at what this diagram illustrates and summaries in bullet points;

Non Provisioned Devices just managed by SCCM or perhaps nothing at all
They then are passed to SCCM for which a Task Sequence is deployed which sets a base image, and then runs the Autopilot import scripts which add the hardware hash to Autopilot.
The machine is then applied to the Autopilot profile and auto-enrolled to Intune.
The machine is then joined to the Azure AD
The endpoint is ready to go

Now Provisioned Devices on the other hand would already be endpoints which are already managed by Intune & Autopilot already, so these can simply be reset if to go through the whole process again.

This diagram gives more of a high level idea of how the overall cycle looks. There are indeed many factors in between the lines (or connectors in this case) which make up the whole workflow. But we will get more into this once we get deeper into the series.

Where We Want to Be

So here is a diagram which gives us an idea on where we really want to be;

So to summarize this in bulletpoints;

Provisioned & Non provisioned devices are logically one of the same
One process evaluates them and gets them into the auto-enrollment & Autopilot directly.
Machines are joined to Azure AD
Machines are managed by Intune & Autopilot.
Endpoints are ready to go.

So the aim here is that SCCM is no longer the go-between to prepare devices which are not known to Intune. The purpose behind this is in the future you will need more of a justification to move away from SCCM and if you are already in a co-managed estate as well as it participating in device provisioning it will be even harder to approach this scenario.

So we want little to no difference between provisioned & non provisioned devices where they use the exact same process which ensures they are managed all round.

How do we achieve this

The most simple answer you could suggest is to simply hit Reset This PC! This would work for provisioned devices of course. But we want to take a look at the devices which are not provisioned in this case.

There are several factors which we have to take into account when it comes to Non provisioned devices and this is addressed by two questions ultimately;

Are the devices company standard ready
If not how do we get them company standard ready

Definition of Company Standard Ready

This can be any criteria that you wish. But the most common tend to be the following;

Windows 10 OS Build
Business Critical Applications
Business Critical Security Baselines
Hardware Standard Checks

Windows 10 OS Build

So for the build versions we can indeed manage these with Windows 10 Feature Updates and specify a version we want to stick to. But lets say we have devices that are not only not provisioned but also a ton of them are a extremely varied mix of build versions. Yes we can just add them to Intune/AutoPilot, but bear in mind they wouldn't exactly be ready to go as per the Company Standard would state. And this is where the convenience of using SCCM would come into play where we can wipe the devices off and place our ideal build.

Now this section may sound like i'm talking against the new outlined process, but that doesn't mean we still cant provision them without SCCM and still address and still use the Windows 10 feature updates.

Business Critical Applications

There's always applications that absolutely have to be on all devices. Once we get to the department or user function level then this will get more interesting, but overall we all have applications where a standard user cannot simply work at all.

Having not only a list of these applications but also defining a priority to each of them is very important because this allows us to understand how we shape and deem what the company standard is.

Business Critical Security Baselines

We all want to stick to a certain security or hardening baseline for all devices which must be adhered to. Whether that's firewall policies, Bitlocker or endpoint security. Having these defined as compliance are absolutely critical to our company standard.

And not just the baselines we want to maintain this going forward to ensure all vulnerabilities are addressed with regular patching as well.

Hardware Standard Checks

These fall into a category of where manual tasks are created. Things such as BIOS settings, asset tag information and various other things which can make or break a device from being managed in a certain way.

For example settings defined in your TPM, or secure boot which may prevent your machine from doing a UEFI PXE boot. These also follow very well in defining a standard.

"In Between the Lines" Process - Next Part Coming

In the Figure 1.2 diagram we showed how we want to group Non provisioned and Provisioned devices together, but what you don't see clearly in that diagram is simply what joins them together and allows them to step through the Modern Workplace Management process.

So this process is what will not only be the replacement for the SCCM dependency method, but also will bridge the gap of addressing many of the points defined in the Company Standard process, but also making sure the device is ready to be on boarded all round.

And we will be going deeper into what this process is in part 3.

SCOM Alert Enrichment Script Template

Tue, 08 Jun 2021 07:56:09 GMT

Know how to enrich your alerts with information that paints a clearer picture!

Introduction

Here is the repository for the SCOM Alert Enrichment Workflow process.

Which essentially is a workflow which allows you to gain the most from your SCOM Monitoring solution by simply enhancing the details of your SCOM Alert details.

How often do we see scenarios in which we have alerts generated and the following issues arise in terms of how you perceive the mapping to be such as;

What CI is actually affected
What area of the business does this affect
Is this a false positive/negative
What priority is this overall
Which monitoring management pack does this derive from

These are a short-list of very frequent "want to be known now" answers in which we may not have access to right away without performing some kind of due diligence to see where this information is.

And that is where the Alert Enrichment Workflow comes into play.

Scomathon 2021

Hopefully you would have been redirected here after viewing the SCOMATHON 2021 presentation which goes into finer detail of the whole Alert Enrichment process. Which doesnt just follow a management pack which you can simply import, but also a several phase exercise and technical/business strategy which can be applied to capture the best and most vital information of your business.

For those who have not seen the Scomathon 2021 presentation this can be viewed here (will be updated once recordings are available of course)

Where to Download

You can download the sealed and unsealed versions of this management pack here .

Modern Workplace Management - Build Endpoints without SCCM

Wed, 26 May 2021 18:48:37 GMT

Learn how to provision devices only with Intune & Auto Pilot

Part 1 - Current Provisioning Methods Available

Introduction

The next phase of when it comes to Modern Workplace Management is the luxury of being able to not only provision endpoints into Intune, but to also be be able to build them using just Intune & AutoPilot....Only!

There are a lot of moving pieces to this structure, so this series will be split up into several parts which will take this solution from end to end to provide a template on how we would want to achieve this going forward.

But first lets analyse a few things, this first part will look through all of the ways in how we would normally provision and build our devices to give a better detailed look into how and why we use these types of scenarios.

Provisioning Methods

Here we will drill into detail all of the current provisioning and building methods when going through the procurement of building endpoint devices, and how they compare to issues which you may face in the long run when it comes to the decision of making the transition fully into Intune/AutoPilot.

SCCM Standalone

How It's Done

So SCCM is the most common and in a lot of cases the most recommended method when it comes to any form of provisioning devices. Some of the key benefits which are (not limited to)

Wiping Devices of current setup
Deployment of a new OS
Relevant drivers for your model
Mandatory scripts to connect to Endpoint Management Portal
Add device to AutoPilot

These are the most common steps which you would see in a task sequence which would be able to provision out all of your devices so that they are Intune & AutoPilot ready.

Drawbacks Going Forward

Whilst this a great way, this method is very dependant on the fact that you would be utilising a base build image which would be deemed as a company standard image (gold/silver image). So the benefit of preparing all devices to follow this structure makes sense.

But its advantage can also be its disadvantage. For instance, if your future plans is to eventually manage all endpoints in Intune, then this method gives SCCM more of a justification that it cannot be retired from your provisioning process and this is where this would really need to be addressed.

Pulling down a sizeable image to deploy to a device can be very time consuming, especially when you take into account that after that it's still not fully complete until it goes through the AutoPilot process.

SCCM (Co-Management)

How It's Done

Very similar to the steps above, however in this case we are using Co-Management which allows your Windows 10 Endpoints to be managed with split affinity between SCCM and Intune.

So in addition whether this is done within your SCCM task sequence or in the AutoPilot application process you would have the SCCM client which would be installed to allow the ability for the device to be co-managed.

Drawbacks Going Forward

Again very similar to the scenario of the SCCM standalone method, however this has more of a complication of dependency. Co-Management is a brilliant way of having the best of both worlds, but it's also always seen as that middle ground for organisations that are more apprehensive on making the full transition into Intune. And this can be down to many many reasons which would go beyond the scope of this series.

Also to efficiently be able to allow devices to be co-managed in this way when having devices prepared in AutoPilot, you would potentially need to factor in how you would get your device in the SCCM environment to be part of the nominated group which is selected for Co-Management, as well as the nominated Azure AD Group which would also have the same selection.

Offline Media

How It's Done

Offline media is a good method when you don't want to utilise SCCM or other building tools such as MDT (Microsoft Deployment Toolkit) to provision your devices for you, and stick to provisioning devices without any other kind of dependency.

The offline media would essentially use a task sequence which you would have created with the tools mentioned earlier and then a local copy of this build would be accessible by removable media such as USB or CD/DVD (be amazed if you had this on CD!).

Drawbacks Going Forward

The real drawback in this case is probably more on the administrative effort to go this route.

If you have a deployment workbench or any kind of build room where preparing a lot of devices is the norm, then having a method which would require you to have a sufficient or even an equivalent amount of removable media to source every device for provisioning could be an absolute nightmare!

Not to mention you can also run into issues where the media itself is not recognized which leaves you being unable to build the device.

Why Build from Only Intune?

After analysing all of the scenarios there is a mixture of areas which can indeed be improved, and with the roadmap pointing towards a more modern workplace management, these are the times to at least be prepared or get prepared for what is going to happen next.

When we think of how imaging machines evolved it went from having a gold standard which had everything, then it turned into a hybrid setup where we would have a base build and simply inject everything.

But now we want to take it up a notch where instead of being able to wipe the entire machine to place another OS or any OS at that, we want to be able to provision and design, customize our standard whilst also prepping for the customers environment as well and this method can save a lot of time when it comes to this process.

We've all witnessed builds of this magnitude with gold images take sometimes up to 3 hours, and that can simply be mixture of the network when it comes to the PXE boot let alone the time it takes to apply the whole task sequence, and then the management of the device in Intune & AutoPilot hasn't even taken place yet.

The method which will be displayed within this series will plan out everything you need to save a lot of time and to no longer be dependant on any other toolset, which will open the doors as to how your organisation will work better in the future (hopefully :) ).

Next Part

The next part of the series we will be going through the high level detail and design on how we would only build devices strictly within Intune & AutoPilot.

We will show how we make a transition from every single scenario explained above so that all areas are covered no matter what your current setup is like.

Modern Workplace Management - Installing Applications in Order

Thu, 06 May 2021 11:34:57 GMT

A solution to stop randomized application installs!

Introduction

When developing a process for your Auto-Pilot one of the exercises we normally have to undertake is going through the base applications that we want to set as a standard for all of our endpoints.

One of the obstacles that we may face is that by default we are unable to set a priority or an order list of all of the applications that are to follow, so if we have applications that may have more of a specific dependency than others this can in turn cause issues.

There have been workarounds where an application dependency has been put in place of another application. It doesn't give you great control over the order as such, but it does allow the administrator to know that a specific application will not be installed unless a specific one has been installed first.

Workaround or Solution?

My suggested method can be categorized as a workaroud, but perhaps with the creativity of a solution perhaps. The aim of this scenario is to come up with a solution which will help us achieve having a say in which applications can be installed first.

As this feature isn't ready by default yet in Intune, it may sit as a a workaround for now.

How to create the Consolidated Application Group

Obtain the Install Files of Applications

First we need to select the applications that we want to add and then we need to be able to obtain all of the installation files so that we can have one consolidated structure. Somewhat like the folder structure below;

To make things flow easier I've removed any spaces from the folders which contain the installation files of each application which we are looking to have.

Create a Script File as the Executable

We will be using a PowerShell Script which will act as the main executable which will then define the order in which we will install the applications and that will be what controls the process.

So below is an example of installing Application 1

What we are doing here is that we have a log file which is then creating locally on the machine with a timestamp, and then the first section is to install Application 1.

Once installed it will then follow exactly where the application has been installed
( Note: Locations and paths can be changed however you see fit)

It will then check to make sure that the executable for application 1 is there and if it is then it will confirm in the log file that application 1 has been successfully installed.

Where the log file is concerned you could place this in the C:\ProgramData\Microsoft\IntuneManagementExtension folder to make the locating of the logs easier and more standardised. The C:\Temp location is really an example of where you can essentially create the log.

Define the Detection Method

Now though we had some sort of a detection method, we ideally want to create more of an official one so that if we are deploying applications which either are exactly the same or perhaps to the same groups in Intune, we don't want to overlap or essentially overwrite this process so we need a unique identifier to mark that this process has been completed.

The best way to do this is by using a registry key to identify its status as well as its successful detection going forward.

So now we will slightly edit the script to show how we would incorporate this into the powershell script and this is now what we have below.

So we have the Create Registry Key line which will set a registry key in which all of the detection methods for each application will be placed in for when we approach them.

So when we look at App 1 , it will run its course of installation but this time it won't just create a log entry to confirm it has been installed but it will now create a registry key value within the key created earlier and will then label this as a value of 1 which would be determined as installed.

If for whatever reason App 1 failed, then we can simply create the same key value but this time mark it as a value of 0 being not installed.

So now if we look at a more complete version of the PowerShell Script this is an idea on how it would look.

So we now see the same types of conditions set for App1 and App2.

Now to complete the overall process we need the final key to confirm that this has actually finished so that the detection method for our Win32 Application passes. So we refer to the last line which will create a registry key value called ApplicationGroup1Status which we would then mark as 1 for being completed/installed.

Build the IntuneWin File

Once we have defined everything (and tested hopefully!) then we should be in a position to convert this into an IntuneWin file.

So first we need to obtain the converter tool which can be grabbed from here .

Once downloaded run the utility and enter the details like the following example below;

This should then create your intunewin file.

Add to Intune and Deploy to AutoPilot

Now that we have our Application Group 1 intunewin file ready we can now proceed to add this to Intune and into our AutoPilot process. So firstly

Log into Device Management
Select the Apps - Windows
Click the + button create a new app.

Select the Windows app (win32) application and browse to the location in which you have created the intunewin file and select this.

Click OK and then proceed to create the app information for your Application Group then click next.

Proceed to the next page and enter the install and uninstall command for your Application Group.

( Note: If you do want to use an uninstall script then you can create one, but for the purpose of this example you can put in a line if you are not using or intend to uninstall this).

Next is the key element of this whole process which is the Detection Rule . We will define these rules using the registry key which would be created in our script example earlier. Configure the detection rule as seen below.

Skip any unnecessary pages and proceed to deploy the application to your group which is mapped to the Auto Pilot enrolment profile, or if using any other methods proceed to deploy the Application Group to where your Auto Pilot profile is mapped to.

Accept these settings and this should complete the process of creating an Application Group and then deploying it through your AutoPilot enrolment process.

Conclusion

This workaround/solution does contain a lot of elements of rewriting or reinventing the wheel somewhat. And perhaps the drawbacks would be we essentially have to define more in the overall process of detecting and logging when it comes to our mandatory applications which we want for our endpoints when going through the Auto-Pilot process.

But for those who require a lot of applications and also are having to go through setting a lot of dependencies to reduce the chance of further randomization, this method does help in taking back control of how you would want things to be done.

Intune - Microsoft Surface Device Configuration Firmware Interface Profiles

Fri, 23 Apr 2021 18:41:33 GMT

Control your Surface Device UEFI Settings for Modern Workplace Management

Introduction

When it comes to preparation within Modern Workplace Management whether you are already fully in Intune or in the mist of moving other from an on-premise/co-management solution to Intune, one of the key things around this transition is the current process in which you use to prepare and build your actual endpoints.

As of most recent of times a lot of us in a real case scenario are utilising UEFI (Unified Extensible Firmware Interface) enablement when it comes to secure booting as well as controlling which internal devices are enabled and much more.

Now when it comes to these settings within the BIOS (though UEFI is on its way to replacing this) though we can automate some of this on an OS level, most of this has been incorporated within a manual process or task list whether this is a pre or post build methodology.

And this is exactly where the DCFI Configuration comes in.

What is DCFI

So DCFI is the Device Configuration Firmware Interface which can be used as a configuration profile to control the following settings;

Whether the User can control UEFI Settings
Enable CPU/IO Virtualization
Set secure booting options
Enable/Disable internal hardware

So this profile can potentially takeover/remove those exact same manual tasks which we would normally have to do ourselves, or lucky enough to be utilising either an SCCM package or an Intune application to perform.

Compatibility of DCFI

Currently DCFI is only really made available for Surface Devices.

The current list is of reference to https://docs.microsoft.com/en-us/surface/surface-manage-dfci-guide

Surface Pro 7+
Surface Pro 7
Surface Pro X
Surface Laptop 3
Surface Book 3
Surface Laptop Go
Surface Laptop 4

Prerequisites

Auto Pilot & Enrollment Status Page

In order to use this profile effectively you will need to use an Auto Pilot configuration profile first and also have your enrolment status page.

You will find if try to create this profile as standalone it may not work, and as this can be applied immediately through the enrolment status page, as soon as your device is ready to go you can double check your device by restarting and booting into the BIOS

( Note: This normally works by holding the volume up button whilst rebooting for surface devices)

Settings Walk through

Allow local user to change UEFI Settings

So this particular setting I decided to create a section for it on its own. Reason being is because if we were using lets say a BIOS password to lock down a normal user from accessing the BIOS/UEFI Settings then this setting will simply override it as this will stop any user from changing any settings if set to None for example.

You can however change the setting so that you can allow users to change any setting which is not configured by this profile but it will be your discretion if this is the correct way to proceed.

Boot Options

For those who are still using PXE for task sequence deployment via Co-Managed topology these settings would appeal as you can use this profile to set the secure boot settings for your endpoints.

How to Configure a DCFI ProfileHow to Configure a DCFI Profile

To configure the configuration profile please perform the following;

Log into your Device Management Console
Go to Devices
Select Windows
Select Configuration Profiles
Click to Create Profile
For the platform select " Windows 10 or Later " and select profile type as " Template "
Find the setting which says "Device Configuration Firmware Interface" then click next.
Create a name for your profile and relevant description then click next.
Select the settings which you require then click next.
Select the group which you will assign the profile to then click next.
Set the Applicable rules which the profile will run against and finish the profile creation.

MEMCM - Migration from On premise to Intune - Part 6 - Migration of Policies

Sat, 10 Apr 2021 08:49:02 GMT

Planning for Client & Endpoint Security Policies

Introduction

The last part on this series will now focus on the various policies within MEMCM, mainly the client policies and the Endpoint/Defender policies.

There is an interesting split as to how client policies can be migrated over but due to the various categories which are displayed in both policies for your MEMCM & then the profiles within Intune will make an interesting exercise to explore the overall mapping.

For Defender policies we will also dive into the migration of these policies and how they fit into a profile around the Endpoint Security area of Intune and how they will all fit overall in your Modern Workplace Management.

( Note: When performing migrations of policies in general its best advised to testing these first on machines which your policies that are controlled on-premise to be turned off so that you able to obtain a true baseline of working policies once migrated into Intune for your Modern Workplace Management. Then once happy with the testing you can make the transition of it being controlled only by Intune )

Client Policy Mapping

Various settings are set by default within your Default Client Settings policy in your on-premise MEMCM. So for all of the settings which are not only applicable but also can be configured within Intune will be explored even further.

The migration of these specifics are not as straight forward as the other parts in this series due to the fact that a lot of the policies have a primary focus on the on premise client settings which utilise the communications & polling methods, however there are some which provide a more overall configuration of the whole Intune/Modern Workplace Management experience.

Computer Agent

The computer agent has settings specifically for your on premise SCCM. For example, if we take the Computer Agent policy as seen below in my default settings.

These settings control the overall branding of the MEMCM environment which you have. So you would expect to see these around your Software Centre & some cases your Application Catalog.

Now within your Modern Workplace Management your Software Centre is essentially your Company Portal whether that be the actual windows application or the web based version. Also not to mention your Auto Pilot/Enrollment/White Glove process as well. So these settings are then migrated into the Tenant Admin End User Experience Section.

Enrollment

Where enrollment is concerned these are covered within the MDM settings, Enrollment Profile pages and also the Auto Pilot provisioning section of your Intune Device Management.

Enrollment settings of course vary from device type, where Android, iOS and MacOSX have their own enterprise level methods of enrollment where you might have a PKI topology and authentication roles defined within MEMCM if you are managing these on premise. There maybe situations in which you were an early adopter of going to the MDM when Intune was transitioning into the Intune Standalone a couple of years back.

Having said that if we look into the enrollment settings on your on premise MEMCM there are options such as allowing users to register devices, now these specific settings are a bit spread out and can be controlled within administrative permissions using RBAC (Role based access control) within your Intune Tenant admin settings.

Hardware/Software Inventory

Unlike the on premise MEMCM settings there are no policies within Intune which can actually set a polling interval to collect the information as this is done in a more hardcoded way.

When devices enroll areas such as Discovered Apps (equivalent to Software Inventory) will collection information ranging from 24 hours (win32 apps) and 7 days for all of the rest. The Hardware Inventory follows the same type of polling. Below are some ideas on how the actual inventory looks like for both hardware and software.

It is to note that the Intune Data Warehouse can also pull down the exact same kind of information for both inventories if you choose to build a custom report or Power BI dashboard from the same information.

Remote Settings

In your On Premise MEMCM you would be able to use the remote control application which will enable you to remotely connect and control a managed asset within your on premise estate.

Intune currently does not have an equivalent as such. But there is a Teamviewer connector which can integrate with your Intune Device Management and this can therefore provide the same service as well as compensate for the policy settings as well.

Other Policy Settings

Other areas such as the following;

Background Intelligence Transfer Service
Computer Restart
Power Management
Windows Updates

These can be controlled from Configuration Profiles especially through the settings view where you can get a comprehensive list of policy categories as seen below

Defender/Endpoint Security Mapping

This section is actually less complicated to migrate over in the sense of that the sections for Endpoint Security settings are situated in one area when it comes to Intune. The policies for this in your On Premise MEMCM however can be somewhat spread but they are still close knit logically within the console it self.

So if we look into the Assets and Compliance - Endpoint Protection section in the MEMCM console we have the following categries;

Antimalware Policies
Bitlocker Management
Windows Defender Firewall Policies

So the mapping side between them to Intune so for example;

Antimalware Policies have the scheduled scan and scan details performed on each device with an agent, in this case we would migrate this into the EndPoint Security - AntiVirus Policies where we can dedicate settings specifically around scans, real time scanning.

As bitlocker was covered in the one of the earlier parts of the series this also has a straight forward migration mapping into the EndPoint Security - Disk Encryption Policies.

And for the Firewall Policies, whether you use these in the perspective of them being mainly controlled via MEMCM , or firewall policies which are controlled from a GPO level and distributed to all of your endpoints, these can also be mapped across to EndPoint Security - Firewall policies within Intune.

These are split up where you can define the settings for your actual firewall profile, and then another one specifically for firewall rules where you can configure the inbound and outbound traffic as seen below.

MDM Security Baselines

These baselines are set by default to add a foundational layer of security for all managed devices within Intune. Now where your policies are concerned within your On Premise MEMCM you may have some configurations which may either overlap, or perhaps be inconvenient to you where your other settings are concerned.

They provide a compliance standard for all devices globally within your Intune, so its best to proceed with caution when defining other policies within your Intune which may conflict with the security baselines.

Below is a screenshot of my MDM Security Baseline (December 2020) which is currently applied to all devices. Now as I have showed I had created a firewall policy it may not override this one, so if specific changes need to be made then you can change the distribution settings of where the security baseline has gone to if you want to build your own custom settings and baselines.

You do have the ability to create your own Security Baseline as well and define very similar settings which would be within policies which you are looking to migrate over into Intune. But please do so with care (Disclaimer at the top of the article)

End of the Series

Thanks again for looking through all six parts of this series and will be creating more around these areas soon which may give a more technical step by step approach on how to achieve these areas of migration.

MEMCM - Migration from On Premise to Intune - Part 5 - Migration of Collections

Thu, 25 Mar 2021 10:56:38 GMT

Transition your Device/User Collections to groups

Introduction

In this next part of the series we will look into the transitioning of the collection estate in your on premise SCCM into Intune to take advantage and map all of your deployments, policies and any additional customized settings you have within your Modern Workplace Management.

This particular part may be pretty straight forward - however there might be some little hurdles when it comes to the structural mapping when going through this transition so will cover as much detail as possible as well as outlining some best practices along the way.

Mapping of Asset Compliance CIs

In SCCM we have two forms of collections;

Device Collections
User Collections

And both types of collections already have existing types where these can be replicated within not just the Intune Portal but also the Azure AD Portal.

Whilst the base perspective of the object mapping is quite straight forward we need to consider the transition and mapping of the actual membership rules that exist in your on premise collection estate.

Limiting Collections

Where you would have limiting collections to restrict which devices can be added into your collection, within Azure/Intune we don't have much of a equivalent to this.

So any device/user will be accessible if wanting to add any CI into an Azure AD group. Having said that the closest to a form of restriction to what you can add to an Azure AD Group can possibly be controlled using an Administrative Unit which can control which users can actually edit and add CIs to the group. But in terms of an actual limiting base for the CIs in the group itself there isn't really one.

Group nesting with groups that have dynamic rules added into other groups are other alternative methods which will be discussed in further details below.

Refresh Schedules

This would apply more to do the query membership type rules discussed further in this article. This particular area has a slight draw back where you cannot set a scheduled time date to refresh groups which as it stands takes about 24 hours to do a complete refresh.

There is a workaround to force a refresh (interesting mapping for Update Membership in SCCM :) ) where you can either edit the name as a whitespace then delete then it will update. This will however trigger the group membership to update immediately where it may take longer in SCCM depending on how complicated your SCCM collection structure is.

Direct Membership

So this is a straight forward add a single device or multiple devices into a collection with no specific rule in place just a straight forward explicit addition to the group so we will look into how its done in the on premise way and then investigate how its done within Intune.

On-Premise SCCM Method: Direct membership works with just creating a new collection and then adding a direct membership rule, or you can right click an existing collection and select add resources.

Intune Method 1: Add Members allow you to perform a similar function to the method described however in this case you have a list of all of the devices of both devices and user accounts

Intune Method 2: Bulk upload: Here you have an option to perform a bulk operation to inject the deployment group with all of the devices. So when migrating from your on premise collection you can grab an export of all of the devices by its UPN name and import using the template provided in Figure 1.2

Query Membership

A query membership in On Premise SCCM uses a WQL query which builds a logical condition as to which devices or users can be dynamically added into your collection.

So we will list out the mapping between the On Premise and Intune methods.

On Premise Method: Query Membership Rule - Create a WQL Query to build a conditional query to structure adding CIs in a dynamic state.

Intune Method: Dynamic Device/User: In this particular setting we set a rule which can detect various different settings on the CIs which we are using to build the dynamic query for. You can see this further in the examples in Figure 1.3 and Figure 1.4 which details a rule for picking up just Windows 10 20H2 machines. In addition we can also run a validation test so that we can see a preview of what devices will/wont be added to the Azure AD group which is great for troubleshooting.

Include/Exclude Membership

In your On-Premise SCCM you will have membership rules such as to Include/Exclude groups from your device collection.

On Premise Method: Include/Exclude Memberships: Here we can select either method and then select the corresponding collection which you will add to these rules.

Intune Method: Group Memberships: There isn't an exclude membership rule but the Group Membership setting is the equivalent to do the Include membership rule which you would see in SCCM.

Best Practice for Deployment & Naming

The best recommended practices for this section are pretty identical to how you would structure your collection designs within your SCCM. But we will try to optimize them for how you would do them within Intune to give a clearer picture.

Clear Naming

Ideally the azure group naming should be clearly outlined so that they can be easily identified for what they look after. For example if we were creating a device collection for Zoom deployments something where an Azure AD group would have a naming convention such as <Application Name>-<Version Number>-<Install or Uninstall> so in translation;

Zoom - 10.0.0.0 - Install
Zoom - 10.0.0.0 - Uninstall

Its important that the name you give the azure AD group has a similar name to the application added within your Intune Application estate.

Cross CI Membership

There might be cases where this is not avoidable, but where possible please avoid this! I've seen several estates where CIs can be parts of deployments where they are part of install and uninstall deployments of the same policy or application.

In these cases you can utilise the exclude group deployment options which setting an assignment which can be viewed in Figure 1.6.

Next Part

The next part we will go into detail on how we perform the mapping and transitioning on the policies migration from client policies and your endpoint security settings.

MEMCM - Migration from On Premise to Intune - Part 4 - Migration of Software Update Strategy

Wed, 17 Mar 2021 15:32:23 GMT

Transform your current software update process into Modern Workplace Management

Introduction

In this part we will go into detail on how to transition your on premise software update solution into being managed completely by Intune.

How your Software Update Strategy is Currently

For those who are using SCCM/MEMCM and using the SUP (Software Update Point) Role then you may be utilizing the following types of configurations;

ADR (Automatic Deployment Rule) Deployment - Automated schedule deployment strategies like the commonly used Patch Tuesday templates and automatic approval of specific update categories.
Manual Software Update Package Deployment - Synchronization of Windows Updates from your WSUS to your SUP role which will show within your Software Library Software Updates View.
Offline Servicing (Image Patching) - Software update patching directly to your image to obtain the latest updates.
Task Sequence Build - With a standard task sequence you may have a step which includes the Install Software Updates step which will just apply the latest updates to your newly built machine.

And also in smaller cases you may just be using WSUS as standalone with no integration with SCCM that will be your patch management strategy.

With all this being said we will show how to transition into Modern Workplace Management and how we can follow a similar service or enhanced alongside optional additions around your patch management.

Existing Windows Feature Update Deployments

This can fall into a couple of categories depending on what your strategy is when it comes to upgrading the build version of your Windows 10 Endpoints.

Software Update Synchronization is a normal method where you can deploy this in it's own software update group, though this can come with its own challenges and alternative methods have been used where you can use the feature update in either a Task Sequence Upgrade or even an application - article for that can be found here

Transition to Intune - Update Rings

ADR Deployment Rule & Patching Migration

Following on from the ADR Rules, if we want to replicate a similar structure we can achieve this from creating a Windows 10 Update Ring which will allow us to deploy updates for Windows 10 (Windows 8.1 also if applicable) automated & on schedule.

So taking for example the template which can be used for patch tuesday here is an example of how the Update Ring would be configured for this.

Here is a mapping between the ADR Rules and the Windows 10 Update Ring configurations.

ADR Properties
ADR Property: Software Updates - In this tab you have the ability to select the type of categories in which updates you will automatically approve and then deploy to your managed clients within SCCM.
Update Ring Settings: Microsoft Product Updates: This is simply a toggle between yes and no which allows your update ring to pull in the latest updates. The updates are brought in by a cumulative basis so you aren't able to specify categories that you would wish to exclude from your rule.
Update Ring Settings: Servicing Channel - Here you can specify which channel you want to choose from your updates with semi-annual where you obtain updates as soon as Microsoft releases them or windows insider where you can get updates earlier before official release..

ADR Property: Evaluation Schedule: Here you can specify the date and time in which the ADR rule will run and by further configuration will allow you to specify if you want it run on that schedule or if you want it to run after a manual software update synchronization has been triggered.
Update Ring Settings: User Experiences: In this section we can dictate the schedule but in more finer detail for not just the date and time but specific weeks and times within the month which can help us replicate specific templates such as patch tuesday. We can also specify deferral periods and deadlines similar to an actual deployment within SCCM when setting up for a collection deployment.

Windows Feature Update Deployments

In order to migrate the process of how you deploy your feature updates, there is now a separate ring in which we can control this and that is through the Windows 10 Feature Update Ring .

The Feature Update Ring and normal updates ring work together where though the normal updates ring can deploy updates for both the rest and feature updates, the separate ring specifically for feature updates will freeze the specific version which you want to limit the feature updates for.

In this case we have now limited the feature update to deploy the Windows 10 20H2. Now when we refer back to the settings in the Windows 10 Update ring we can now set the deferral period which will work specifically with the feature update ring.

Task Sequence Build/Offline Servicing Migration

Hopefully at this point you are moving away from the traditional physical builds and utilizing auto-pilot :) In this case we want to design the workflow so that any newly enrolled machines are brought into the update patching rings applicable to them.

So if we have the update rings configured above, we can have an Azure AD Group which is specifically tied to that Windows 10 Update ring, so any new machines brought into your Intune environment will receive the correct patching.

Next Part

The next part we will focus on the Device/User Collection migrations and the planning of migrating them into Modern Workplace Management

MEMCM - Migration from On Premise to Intune - Part 3 - Migration of Software

Sat, 06 Mar 2021 12:33:48 GMT

Planning the migration of your Software Library Estate

Introduction

In this next part we will investigate on how we will go about the planning of the migration of the whole Software Library estate into Intune.

There are many categories within the Software Library which have different variants on how we will go about the migration and also equivalent options.

Not every single object within your Software Library could be applicable to migrate into Intune so therefore we need to outline what can be migrated, and relevant alternatives which provide the same service to standard for your managed endpoints.

Categories of Your Software Library

Just to analyze once again we have various types of objects such as

Applications
Packages
Task Sequences
Drivers
Script Packages
Configuration Baselines

We will go into more detailed specifics in each type listed above.

Applications/Packages Migration Planning

In the very first part of the series we did cover a good amount of information so we will do a slight recap but this time we will cover more basis on how the application migration planning will go.

Windows 10 Endpoint Applications

If we look at the basis on how we would migrate our on-premise estate to Intune in most cases our application estate would consist of various file type categories with the main ones being;

.MSI
.MSIX
.Appx

These are ideally the the type of applications which we want to be dealing with when it comes to the planning of how we will carry out our migration of them into Intune for Modern Workplace Management.

But in real world scenarios though we have Packages we sometimes have applications which can indeed use script based deployments which can either consist of using a .exe file extension or utilizing an actual script file whether that be PowerShell, VB Script or command line/batch file scripts.

Application Categories
So firstly we need to establish the different kinds of applications which we can actually migrate. So to summarize here are the different ones;

Line of Business Applications - This category is for the main file extensions above being .msi, .msix, .appx. These are very straight forward imports very similar to what you would see when importing applications with the same file extensions in SCCM.
Windows App (Win32) - This category is for the script based packages which utilize a .exe or other script files which will act as your main installation command. And very similar to the on-premise SCCM you will need to configure the detection methods which will check if the application already exists on your endpoint and also to detect a successful installation. These applications however are not something you can just import into your Intune portal as they need to be firstly converted into a .intunewin file extension. One of the best tools you can use for this can be found here

Mobile Device Applications

For those for are managing mobile devices within your on-premise SCCM such as;

Android (.appk applications)
iPhone/iPad (.ipa applications)
Mac OSX Devices (.intunemac applications)

Then there is a chance that you will be able to migrate these applications over to Intune as well. .IPA & .appk applications can be migrated within the same category as the Windows 10 applications within the Line of Business Applications category.

For the Mac OSX devices again these will have to be wrapped very similar to the script based packages for Windows 10 to obtain a compatible file to import with a .intunemac extension. The best tool to use for this can be found here

Task Sequence Migration Planning

A lot of areas to cover in this particular section which will be able to replicate a similar process to how your on-premise Task Sequences would work when transitioning into Intune.

So Windows AutoPilot is really the method of building machines and as well as enrolling of new devices, but what we will do is have a side by side analysis on what you would typically see in a standard task sequence and output an equivalent where your entire process in Intune will be able to replicate the entire process of your task sequence to maintain the same standard of service which you were providing before.

Of course we will take the actual building of the OS and machine itself out of the equation but look more into the process of the post OS tasks which we can make Intune responsible for to complete the build. And of course where you may have other Post OS install steps which are more customized this guide may have pointed out areas in which you can use these, and perhaps even further extended into different Intune policies which will be discussed in another part.

Task Sequence Task/Group: Join Domain
On-Premise Solution: So within SCCM a domain join step would be included by default to specify the domain and OU which the machine will be placed into. More advanced techniques involve using Task Sequence variables in the pre-build section of your Task Sequence which would be able to detect the type of machine and going by certain criteria would determine how the machine would join the domain and where it would be placed.
Intune Solution #1: Domain Join Profile - Just before I put the obvious option of an AutoPilot profile thought I would pick on this one for a moment. The domain join profile will do quite a similar step to the on-premise one however here you can define a prefix for your machines in this profile by default as well as configure the domain it will join and OU it will be placed in. Where SCCM would use a collection to deploy its task sequence Intune would use an Azure AD group to be assigned to this profile.
Intune Solution #2: AutoPilot Deployment Profile - Of course we can't forget this one. We might find this will probably be the answer to all of them as the Deployment Profile will handle the configuration of everything, but have to take into account other components such as the Enrollment Profile Page which would essentially be a close replication of what you would see on a Task Sequence splash screen during WinPE phase. The reason for putting this option second is because in reality your machine will join the Azure AD Domain by default, but if wanting to add an additional layer of where you want them to be placed a Domain Join Profile is what you will require. You can however use the Deployment Profile settings to apply a device template which will allow you to construct the naming convention for new devices once they go through the enrollment process.

Task Sequence Task/Group: Install ConfigMgr Client
On-Premise Solution: So within SCCM this step would be to install the Configuration Manager Client and also the Software Centre, which is what would be used to be able to be able to receive deployments & policies from your management server.
Intune Solution #1: Company Portal Deployment- Within your Modern Workplace Management you wouldn't be using a client as such, however you may want to use the Company Portal application which is the equivalent to how Software Centre works. Now it's not absolutely necessary as you can still get deployments from Intune without it as well as a web-based version of your Company Portal. But in a more user-friendly perspective and as well as bridging the gap from an on-premise solution to Intune this maybe best. There are further bits to this as you would ideally need to obtain this from the Windows Store for Business to grab the correct license which will sync into your Intune.
Intune Solution #2: AutoPilot Deployment Profile - Used in conjunction with the solution above we can point the Company Portal deployment to the Azure AD Group which contains all of the AutoPilot based devices or devices in general. In a case of a Task Sequence scenario, the Auto Pilot would install this application during your enrollment setup.

Task Sequence Task/Group: Install Applications/Install Packages
On-Premise Solution: So within SCCM the step to Install Applications would comprise of a list of applications which would have to be enabled to be eligible to be installed in Task Sequences first and then added into the step. You can also use Task Sequence or Collection based variables to determines lists too. Install Packages quite similar where you select the package then the corresponding program.
Intune Solution #1: Application Deployment - In this case as we covered earlier deploying applications already groups SCCM Application & SCCM Package based applications into one group. So essentially we can import those into Intune (with converting script based to .intunewin of course) and deploy them to Azure AD Groups.
Intune Solution #2: AutoPilot Deployment Profile - In conjunction with AutoPilot we can deploy those same applications to the Azure AD Group which AutoPilot devices are part of so they are a prerequisite before auto enrollment has been completed.

Task Sequence Task/Group: Install Software Updates
On-Premise Solution: So within SCCM the Install Software Updates step really just initiates a sync between the built machine and your SUP (Software Update Point) to obtain the latest updates applicable to the machine.
Intune Solution #1: Windows 10 Update Rings - In Intune we can create Update Rings where we can customise the type of updates that will come through as well as there deferral periods and the type of update channels it will use. These profiles can be deployed to an Azure AD Group
Intune Solution #2: AutoPilot Deployment Profile - In conjunction with AutoPilot we can deploy those same update rings to the Azure AD Group which AutoPilot devices are part of so they are a prerequisite before auto enrollment has been completed.

Task Sequence Task/Group: Bitlocker Encryption (Optional)
On-Premise Solution: So within SCCM you will have a group with a couple of steps to prepare Bitlocker via its TPM module and then proceed with the Bitlocker encryption and where to play its recovery key.
Intune Solution #1: Disk Encryption Profile - In Intune in the EndPoint Security Window we can create a Disk Encryption policy for Windows 10 utilising Bitlocker where we can configure the settings and to which drive the encryption will be applicable to whether it be to the System Drive or other drives. The policy can be deployed to an Azure AD Group.
Intune Solution #2: AutoPilot Deployment Profile - In conjunction with AutoPilot we can deploy those same policies to the Azure AD Group which AutoPilot devices are part of so they are a prerequisite before auto enrollment has been completed.

And as for other Post OS install bits (where enrolment completes) any policies which you will configure in Intune maybe applicable to your machine also. For SCCM you would have your default client policies amongst other custom based policies which Intune can also do which may also answer for other bits in your specific Task Sequences depending on the kind of setup you have.

Drivers Migration Planning

Where your drivers are concerned for your hardware, they follow a similar pattern to where the AutoPilot is concerned where the responsibility is down to the OS build provided to your hardware provider for when it comes to building the machines.

Having said that there are other ways in which some drivers can be migrated or at least an alternative in which it can transition into your Modern Workplace Management.

Windows 10 Update Rings
Windows 10 Update Rings have the ability to include Windows based drivers which would be applicable to your hardware vendor which will allow your endpoint to utilize the best and latest drivers available to be applicable with your current Windows 10 version.

Depending on the level of drivers which are covered by your initial OS build before AutoPilot comes into play, the windows updates will really either maintain or update the current drivers that you have. Below is a screenshot showing where the settings is located.

Application Based Driver Deployment
If there are more specialized types of drivers which you want to include in addition to your initial build, another way for this to work is to perhaps build a Windows App (Win32) based application which would be part of a script based installation which can be applied specific to your Azure AD groups which can have rules to only include a specific vendor.

For example if we take a Lenovo laptop (oddly enough typing this on one :) ) you have applications such as the Lenovo System Update utility. We can have this where the application can be installed on only Lenovo laptops which can obtain the latest drivers. In some cases alot of customers have a package in their on-premise SCCM which may already contain a tool such as this specific to their hardware vendors and they can in turn be migrated into Intune.

Script Based Deployment
Not exactly as described above though we are talking about a script based application. In this case I'm speaking of a method in which you can run a PowerShell Script package which can perhaps trigger an action on a tool which may already be installed on your laptop such as a Lenovo System Update utility, similar to how a scheduled task would work.
This would be great within a proactive remediation script task which we will cover more in the next section.

Configuration Baselines/Script Migration Planning

Configuration Baselines and its child items of Configuration Items cannot exactly be directly imported into Intune, but they can indeed be replicated into Proactive Remediation Scripts.

In SCCM you would have configuration items into a configuration baseline, however in this case the proactive remediation script is just one object which does make it easier to manage the objects going forward.

Below is a screenshot of the settings of a remediation script to outline the similarities of a configuration item within SCCM.

PowerShell Script packages also have the same caveat. But easily can be replicated as long as you export the powershell script used in your on--premise SCCM packages and then upload these into an Intune PowerShell script.

See below the configuration of a PowerShell Script which obtains a list of Windows 10 updates.

Next Part

The next part we will go deeper into the configuration of the Software Update migration where we will analyze how to bring over your update policies and automatic deployment rule configurations.

MEMCM - Migration from On Premise to Intune Part 2 - Migration of Devices

Tue, 23 Feb 2021 18:38:58 GMT

Part 2 - Migration of Devices Planning

Introduction

This part is a continuation from the first part of this series which can be viewed here

This next part will be around how we will look at the migration of our devices from our current on-premise estate and into Intune, of course this will be very dependant on which scenario that you are facing which we will go into more detail on each one.

Scenario 1 - Windows 10 Endpoints only in the Estate

In a perfect world this is exactly where you would want to be with only just Windows 10 endpoints within your on-premise estate which will be the easiest form of transitions when it comes to the migration into Intune.

Having said that we need to analyse what is going to be the most efficient solution for this. We have the following options;

Auto-Enroll Devices by AutoPilot
Auto-Enroll Devices by Hybrid AD Join
Manually Enroll Devices

All of the options listed above will indeed depend on the size of the estate that you have, at the same time I'm pretty sure even if it were about 50 devices you still probably wouldn't want to enroll these devices manually :)

So lets explore each of the options in which we have so that we can lay out the perfect option which is appropriate for your needs
( Note: Step by Step guides on how to perform each option will be coming soon)

Auto Enroll Devices by AutoPilot

Whilst we might be jumping the gun in terms of discussing the Auto Pilot process now it is quite applicable in this scenario as well when it comes to the process of auto enrolling devices.

Auto Pilot is not only just the process of on-boarding brand new machines which may or may not be sent to you from your hardware vendor, but it is used to automate the whole workflow of an automatic profile mapping which consolidates the process of an OOBE (Out of box Experience) for the user as well as the machine. This also is true within doing the same when it comes to auto-enrollment of devices too.

The main prerequisite of this is to of course have your Auto-Enrollment settings configured as seen below

Most tend to use all for the MDM user scope but using the "some" option is better if wanting to have more authoritative control on what or who is being enrolled into the estate.

With various steps in between the other major part is how we obtain the devices information to get them enrolled via AutoPilot and this is where the hardware hash information comes into play. These are the bits of information that are normally required when pre-staging AutoPilot devices before and when a user first logs in.

Auto-Enroll Devices with Hybrid Azure AD

This option is identical to the option above as it also utilises the whole AutoPilot process, however it differs due to this is an option if all of your machines are joined to an on-premise AD only, and having your machines then joined to an Azure Active Directory in an offline onboarding type of method in turn makes them hybrid joined machines.

In addition to the Auto-Enroll Settings you will also required something called an Intune Connector, which if you are familiar with how a Power BI On-Premise Gateway works then this is essentially the same in which it provides a gateway between your On-Premise managed machines and to the Intune Management in Azure.

Various more components come into play such as Domain Join profiles and also delegation configuration required on your on-premise Active Directory so that AutoPilot machines can be made/added to a container which you will specify which will essentially be owned by the machine you will decide to be your Intune Connector
( Note: Server has to be at a minimum of Windows Server 2016)

Manually Enroll Devices

Assuming you don't have a huge estate then perhaps the manual way of enrolling a device could be more appropriate for many reasons. This process really involves you adding the machine to the enrollment of Intune Management by associating it to the user account which indeed has the correct Intune Administrator access to enroll it to Intune.

This is done through Work and School account access which I will detail the step by step on how to perform this.

Do a search for "Access Work or School Account Settings"
Click on the found option which should be found in System Settings
Click the Connect option to get the following screen below

Here you can join your machine straight to your Azure Active Directory domain and then you will have the machine enrolled into Intune as an Azure AD Joined machine.

On the other hand you also have the option of enrolling the device only into Intune and not being Azure AD joined but this perhaps maybe beyond the scope of this article and also wouldn't recommend that option especially if wanting to have a fully migrated estate into Modern Workplace Management.

It's also worth pointing out that manually enrolled devices can also have an AutoPilot profile deployed to them which will allow them to be converted into AutoPilot devices so they can be overall managed by the whole AutoPilot process including the support of device resets to go into the whole OOBE process again.

Another method which can also be used is the Company Portal application which can be used to enroll your devices into Intune, though category wise they may show as a personal device rather than a corporate device which you would most likely get if the machine was Azure AD Join based.

The category itself can be manually selected but for auditing purposes it could be a good method of organising how devices are coming into your Modern Workplace Management.

Scenario 2 - Windows 10 & Windows 8.1 Devices

This scenario may capture a bigger audience than realised, as through many reasons and in the real world scenarios we still might have a mixture of legacy devices which would evolve around the Windows 8.1 devices and possibly as low as Windows 7 (sure hope not :) )

But in these situations there isn't really an option to migrate every single device into Intune as the rest of the estate will be more than likely left out of the migration

So in this scenario there are two options;

Upgrade the machines to Windows 10
Co-Management between On-Premise SCCM & Intune

Upgrade machines to Windows 10

This does sound like a no brainer :) but we do have to take into account there are a lot of estates which do have a good amount of legacy devices for many business reasons that they may perhaps play business critical roles and may not always be as simple as blowing those machines away.

But if wanting to move forward then at some point these would indeed need to be tackled, with another point of them being out of support as well.

Some Windows 8 devices can be managed by Intune if they either RT or sustaining mode but that's as far as it will go.

Co-Management Between On-Premise SCCM & Intune

Co-Management is the perfect blend to share the responsibility of device management between both Intune and SCCM. In this way your Windows 10 Endpoints can essentially be managed in Intune whilst the legacy based devices can be primarily looked after within the SCCM environment.

Another thing to take into account is the policy splits, as in if you are going to migrate your Windows 10 devices then you will need to have a form of replication in which the same coverage of policy compliance can be carried out through Intune the same way it had been successfully done through your on-premise SCCM Environment.

Co-Management works with nominating a collection which will contain the Windows 10 endpoints which you will use to be managed by Intune and SCCM, and these would also be online with an Azure AD group that also have the same machines synchronized within the Azure AD.

Scenario 3 - Windows 10 Devices and Mobile Devices

This is also another great scenario where we can bring all devices across for Intune Modern Workplace Management, especially when you have an MDM estate. Having said that just like Windows 10 needing to be on a minimum of build 1709 mobile devices also have its minimum requirements before they can also be managed by Intune

Android Devices - 5.0 Minimum
iOS Devices - 12.0 (13.0 for iPads) Minimum
Mac OSX Devices - 10.13 Minimum

Also there are different options for enrollment for each device category.

Android Enterprise
Apple MDM Push

Android Enterprise

There are actually two ways in which you can enroll Android devices.

You can enroll devices by going through the Company Portal website which you can obtain from the google play store which is very similar to how you can enroll devices manually within the Windows Endpoints.

The other method is the Android Enterprise way which is also managed by google play. This is essentially where google play is managed by an email account which would be an Android Enterprise administrator account and this will be used where those mobile devices would also be signed into the Google play with the same email address and that would enroll your Android mobile devices into Intune.

Apple MDM Push

With iOS and Mac OSX devices you won't be able to enroll devices into Intune using the Company portal way, as the main prerequisite requires an MDM push certificate which each device needs to have installed before it can be officially managed by Intune.

Similar to the Android Enterprise method you will require an Apple ID which would be the administrative email for which would be used to enroll the tenant into Intune.

Next Part

The next part we will look into the Software Library estate in preparation of how you would look at the planning of migration of various applications and types

MEMCM - Migration from On Premise to Intune

Sat, 13 Feb 2021 13:53:40 GMT

Part 1 - Planning of the Migration Process Mapping

Introduction

The process of migrating over from an on-premise SCCM infrastructure to Intune for Modern Workplace Management can be quite a huge task depending on your current SCCM infrastructure and also specific roles in which it plays.

In one of our previous article series we covered the transition of having a Co-Management environment in which you can experience the best of both worlds with having policies configured between both SCCM and Intune and having the ability to change the affinity between both environments from whether being shared 50/50 or controlled completely by one or the other.

We find that Co-Management or other hybrid type solutions are indeed a great way to have the best of both worlds but it can also be one of those options in which organizations are apprehensive to move over completely to Intune and this can be for various types of reasons which prevent them from making this decision.

This article series we will cover everything you will need to take into consideration from the planning and all the way to the actual migration of your current setup.

This first part shall discuss various categories to consider when planning on how you will migrate into Intune but more on the perspective of the objects and the roles your on-premise environment plays.

This series does take into account that you do indeed hold the appropriate licensing to manage your Intune with an E5 license assigned to your Intune Administrator account with both Intune Admin & Global Admin privileges.

More licensing information can be found here

Migration of Devices

This is perhaps the biggest deal breaker in a sense when it comes to making the decision of actually moving over into Intune.

Not necessarily thinking of the devices being synced into Azure AD or your AD being solely within Azure but more about the mixture of your endpoint estate in your on premise SCCM.

So Windows 10 are really the only endpoints in which you can manage, there are exceptions for some versions of Windows 8.1 (supported devices can be seen here ) so if your estate has a mixture of OSes from Windows 8.1 (hopefully not below!) then you may face an issue in which you would potentially consider a migration in which devices would be left out from Modern Workplace Management hence why many organisations would stick or investigate a Co-Management perspective where they can add their Windows 10 endpoints to a desired collection which would be shared between your on-premise environment and Intune.

Migration of Applications & Packages

This section perhaps one of the most important, well they all are very important but perhaps more on the bulk-ish side of the migration. So things we have to take into account would be the following questions;

How many applications do we have
How many packages do we have
How many are using a .msi
How many are using a .exe
How many Windows 10 based apps do we hold (.appx)

So knowing how many applications and packages in total is one thing. The next part which can be somewhat challenging for administrative effort is where the file types come into play.

For example .MSI applications are easier to import into both Intune and SCCM due to the metadata which it holds which is able to complete the process a lot quicker, and these are seen in Intune as Line of Business Applications.

Windows10 based applications which have the .appx file extension also carry the same less burden appeals like the ones above as these are seen as Line of Business Applications. Where Office applications are concerned these can be offloaded to where we can create Office 365 Applications on demand with configurations of any channel as well as the type of products we want to include (subject to licensing such as Visio and Project) which are categorised as Microsoft 365 Apps for Windows 10.

But of course we have the packages within our on-premise SCCM which are considered legacy based, which was how SCCM 2007 used to handle all applications in this same format which now update has its caveats. So packages can commonly utilise script based installs as well as .exe file extensions and these would need to go into a process which they will have to be converted into the new file format called .intunewin and these are categorised as Windows App (Win32) applications .

Can be potentially the most time consuming section depending on the software library count which you have.

Migration of Collections

For our main source of grouping both devices and users we would perhaps want to either maintain our current tree structure of collections or may even want to improve them.

Within your Azure AD or Intune Administration console you are able to create groups which can be either user or device based and also explicit members (Direct Membership for On-Prem) can be assigned or dynamic based rules (Query Membership for On-Prem) which we would be able to create a criteria for which devices can be added into the group.

Collections are also our logical group objects within our on-premise SCCM environment for deployments of applications, programs, software updates, Task Sequences etc.. so we want to ensure that we are able to migrate a healthy managed collection structure that already exists or even take the opportunity to improve this.

Migration of Client Policies

Alongside the default client policy in which in most real-case scenarios we ensure its priority is the lowest, we also may have a lot of custom client policies with many settings which can apply again to the desired collections that are also based on another filtering criteria.

Here we want to ensure that our custom policies will be indeed brought over but we must take into consideration that a lot of settings which are specified in these policies may not be fully covered in just one profile within Intune as there are various other profiles it can be split into such as;

Compliance Policies
Configuration Policies
Conditional Access Policies (some small elements)

There are also migration processes for your GPOs (Group Policy Objects) which we will cover further in another part of this series.

Migration of Baselines & Scripts

This section will be quite interesting with how we would perhaps looking into the migrations of these.

There are various areas in which configuration baselines can be split across to formulate and keep the same structure to provide what is needed here. So within the Intune environment we have the PowerShell Scripts which we can use for our script packages which can be deployed to your Windows 10 endpoints where needed.

The more interesting one would be the configuration baselines, which though partly can be done within the PowerShell Scripts section, but they may also be more fitting within the Endpoint Analytics features of Intune in which we can not only consolidate them into proactive remediation packages and have a breakdown of how many would be remediated and failed, but then there is also the functionality of where we can create baselines to represent the progress of them which in fact provides a small functionality of historical data to compare both with, as opposed to perhaps a Data Warehouse role which would have to be installed on your on-premise SCCM environment.

Migration of Task Sequences

A form of very interesting creativity here in how we would go about the process of migrating the Task Sequences into Intune.

So a Task Sequence being a sequence of various of actions of course installing the operating system as well as being followed by various settings, Pre OS conditions, Post OS conditions and of course software. Now of course Intune doesn't utilise Task Sequences as such, but this perhaps where Windows AutoPilot would come into play.

Whilst the OS would be installed either by your specific vendor of choice, third party or even done internally, what we can do is control all of the settings, configurations and software deployments. There are various areas in which this would split up such as;

Autopilot Deployment Profiles
Enrollment Pages
Apps
Conditional Access Profiles
Compliance Profiles
Windows Update Rings

There is a great amount here in which we can provide a consolidation for your task sequences to ensure a smooth transition.

Migration of Antimalware Policies

Similar to client policies above we now take into consideration of the Antimalware configuration in which your devices are most likely managed by Windows Defender from your on-premise SCCM.

Custom Antimalware policies as well as further configurations for firewalls, device security (Bitlocker) and more can be migrated smoothly into the Intune environment with very similar categories as well as utilising Defender ATP.

Migration of Software Update Management

This process is perhaps a lot more simplified when it comes to the migration as we wouldn't need to migrate the physical updates, but more around the ADR (Automatic Deployment Rules) we can bring over if they are available within your environment.

Software updates are now handled by rings respectively for normal Windows Updates and also for feature based updates when wanting to upgrade the build versions of your Windows 10 endpoints.

Migration of Reporting

Various reports are available by default in both environments (Intune & SCCM) so it will really be a case of which reporting information is vital to you. Besides the default reporting and charts we have within our Dashboard, we also have the ability pull out further information by using the Intune Data Warehouse source in which we can create a dashboard either through Power BI or SSRS using its OData feed.

There are a lot of box standard reports used within SCCM and don't think many organisations have used all of them, where in this case it leaves the option available for those who may want to pick and choose which information they do want to see which can be shared by some of the default Intune reports and the data source of the Data Warehouse.

Next Part

The next parts of these series we will explore each of these sections in a more finer technical and design detail on how we would make the migration process smooth from your on-premise environment to Modern Workplace Management.

Modern Workplace Management - Endpoint Analytics

Fri, 05 Feb 2021 18:14:15 GMT

Managed Windows 10 Device analysis reporting for compliance

Introduction

When managing your estate through Modern Workplace management, specifically with Windows 10 Devices we want to try and gather as much data as possible, especially when it comes to the reporting of significant data in which we can understand all of the performance of our managed windows 10 estate.

The best way to get control and take advantage of this is the utilisation of Endpoint Analytics.

Endpoint Analytics gives the Intune administrator the ability to collect data from your Windows 10 devices such as;

· Start-up Configuration

· Recommended Software for devices

· Proactive Remediation

Endpoint analytics is somewhat similar to how you would have seen desktop analytics which is more specific to on-premise devices as opposed to where Endpoint Analytics is more driven towards Intune/Co-Management type of scenarios.

Before we get into the configuration of Endpoint analytics lets look into each section closer and see how it looks at a Windows 10 device managed within Intune.

( Note: Currently this information is not available or exportable through the Intune Data Warehouse yet as it’s still in a preview mode. But hopefully this will be available soon!)

Start-Up Configuration

The start-up configuration data shows deeper analysis on a managed device around the following categories;

· Average Start-up Time (Seconds)

· Boot History

· Sign-in History

· OS Restart History

Here is an example of how this data is visualized when captured from a Windows 10 Device. Below is a start-up score which is measured on a baseline which can be configured and measured against other baselines to ensure that all of your devices are aligned with the best compliance results.

Another window which shows even more detailed analysis can be seen below when browsing to your managed asset, which are focused more around the boot history detailing a breakdown of the duration as well as what kind of events had caused them.

Also another chart is one based around the sign-in history of your endpoint.

In addition you also get to look into the start-up processes which can also impact the following events above by displaying a top 10 and OS restart history.

Recommended Software

The recommended Software section shows a summary chart of all of the devices which meet the requirements to be managed within Modern Workplace Management and is categorised for how many meet them such as;

Each bar represents how many are running Windows 10, Azure AD Domain Joined and enrolled.

The bar below is for devices which have been enrolled through the Windows Autopilot where currently there are no devices enrolled currently.

Proactive Remediation

This to me is a very exciting feature (will have more articles around this soon) in where we can deploy remediation scripts and fix any issues on demand or even on schedule.

Now if you are familiar with both Configuration Baselines as well as script packages within SCCM/MEMCM then you will have a good idea on how these work as they have a combination of both worlds where you need both a detection and remediation script.

If we look at a couple of example proactive remediation packages below;

One specific one we can concentrate on is the "Update stale group policy" which is a very handy one for when devices have GPOs which may have not kicked in or applied, and these can sometimes cause issues especially when working with SCCM if certain GPOs play a vital role in your infrastructure (will cover GPO Management in Intune at a later time)

I have various ideas in which type of remediation scripts can certainly be of help, for example kicking off hardware inventory runs if running a co-managed environment where certain devices maybe more offline than usual all the way up to controlling third party updates for various vendors (another which I will cover in a later article)

How to Setup Endpoint Analytics

The enabling of Endpoint Analytics is quite simple, all you need to do is the following;

Log into your Device Management console
Go to Reports
Go to Endpoint Analytics as per Figure 1.1
Go to Settings
Click the "Intune data collection policy"
OPTIONAL - Click the "Configuration Manager data collection" if you are using a co-managed setup.

Endpoint Analytics Baseline Configuration

Earlier we spoke about baselines and how they work in terms of the comparison of other scores on specific reports. Well we can create various baselines by doing the following;

Log into your Device Management console
Go to Reports
Go to Endpoint Analytics as per Figure 1.1
Go to Settings
Go to the Baseline tab

Baselines are great where we can set a measurement on what targets we want our score reports to achieve on a regular basis in order to maintain a certain standard.

So when you click "Create new" and give your baseline a name it will then register the current score of all of the categories which we had mentioned earlier. And for good measure we can also alter the baseline regression threshold which allows us to state how much percentage we will allow the score to go under and still be in line in order to measure realistic expectations.

So now when we go to a page to view the latest analytics you will now see this.

Now you can see there is now a comparison score which makes more sense to the administrator so that you can see how the scores tally up, in this case it is like for like.

Building Power BI Dashboards with Intune Data Warehouse

Wed, 20 Jan 2021 18:03:48 GMT

Get Data from your Intune Device Managment

Introduction

When it comes to the available reports which you see in your Intune Device Management console, you may not get a wide coverage of everything you would want to see, especially when it comes to more information such as the following;

Device Inventory
Full Application Deployment History
Further Device Compliance Information

These needs can be leveraged when utilising the Intune Data Warehouse. It contains all of the historical data within your Intune estate and is refreshed on a daily basis due to the retention periods which it holds.

This can be utilised as a data connection method within your Power BI reports and dashboard which we will show more details of below.

Setting up the Data Connection to Intune Data Warehouse

Firstly open up a new report/dashboard in either your Power BI service or desktop application.
Go to the Get Data option in the toolbar.
In the Search Bar type in "Intune" to find the following listing below

Once you select this option you will get a disclaimer of connecting to this service.
( Note: it is currently noted of it being in beta at this current moment until its final version is released. But will show more detail on the information of which you can obtain from it)

After clicking on the continue disclaimer you will then be given an option to select how many days of historical data that you wish to go back to. Now if you are looking to maintain the latest data the better option maybe to go for 1 day.

But if you are looking to take advantage of a full data warehouse capability then you may proceed to go back further to customize your dashboard even further.

Once you click OK then you will be presented with the Navigator screen which will show all of the available tables within your Intune Data Warehouse. Very similar to if you created a data connection to your SQL Server and Database but without specifying a particular query.

Here you select the appropriate tables which you require.

When ready you can then load the data or proceed to Transform Data so that you can edit any of the information which you have processed from the selection of tables.

Once done you will then have your datasets ready to start formulating reports.

Where relationships are concerned they are pretty dynamic where keys for important data such as Apps, Devices, Device types etc are included in almost every table. So as soon as you apply the changes 99% of the time the correct relationships are formulated which you can see below.

SCOM Management Pack Creator Application - New Features Coming!

Tue, 19 Jan 2021 19:41:41 GMT

Release date and release notes of the new version coming out

The GUI Version is coming

So for all of those that have kept up with the ever-growing development which was the SCOM Management Pack creator in its raw form of the PowerShell Module (presentation still available on https://scomathon.com/coffee-break/custom-mp-creation-with-powershell )

The GUI version is indeed coming and had earlier released some snippets on how it's currently looking and now we are looking to release further details on what it will be able to do plus more.

Upcoming New Features

There will be two versions coming out which will be of a Standard version and also a Premium version.

So currently a lot of work is going into the development of both of them to ensure a clean and minimal (hopefully none) bugs which may come forth after regular use, but also trying to accommodate a majority of features as well defining a smooth end user experience on both version.

Standard Version

Create all components for your Management Pack - Though this might seem an obvious one to certain degree, this is revamped into the GUI interface which will allow a very simplistic and categorised way of creating all components to formulate the whole management pack.
Create Knowledge Articles - You will be able to add more in-depth information with a specific window which will allow you to create a finer detail knowledge article for many components such as Discoveries, Monitors, Rules and many more
MP Summary Dashboard - There will be a window which will display a dashboard like page which will show you a count of every single component which you have added to your management pack before you decide to add to it or finalise the overall build. A menu will also be displayed in which you are able to add various more components at which point the statistics will also follow suit

MP Component Setting View - You have the ability to scroll through all of your created components and view the settings of each one. So once you have created any components you will have the option to look through every one you have created and analyse all settings. This will also be part of the Read Management Pack option which you can analyse a whole management pack to get a easier breakdown of how its been configured. This can be seen in Figure 1.2
Create Relationships and Dependency Monitors
Basic Unix Support for class, discoveries and monitoring
Create Tasks, Diagnostic and recovery tasks
Effective Intellisense Configuration - The application will recommend the best settings where certain options can and cannot work
Custom Module Monitors - Using PowerShell based Monitors
Read Only Mode - Read and analyse a management pack and all of its settings to check all of its configurations

Premium Version

More will be announced on what new features will be available on this particular one - but it will be very advanced and detailed.

Only thing I will say is the below screenshot will give an idea on what to expect from the premium version.

This is a distributed application which was created entirely automated from PowerShell. In this case, the Premium Version will not only be able to create Distributed Applications but you will be able to add components, relationships and create DA's in bulk. But that's all that will be revealed for now :)

Release Date

Expecting the Standard Version to be ready and released in Q1 , which the Premium Version to follow right after. Will be making announcements and releasing more snippets closer to the time once the application is ready to go.

Building the perfect CMDB with System Centre Part 4

Sat, 26 Dec 2020 19:36:11 GMT

New Connections for bespoke tools

Introduction to the next part

On the previous part we spoke of how to focus on the data integrity of all of the foundational components of all of the default connections which would plug directly into Service Manager such as;

Active Directory
System Centre Configuration Manager
System Centre Operations Manager

And with these tools now being more lean with correct data we are then satisfied that we are pulling accurate data to populate our CMDB.

But now in this part we look at the same type of strategy, however we now focus on tools or bespoke tools which do not contain a connector at all and therefore have not only build the connection from scratch but also have to develop the strategy for ensuring correct data is being pulled from it.

This will contain several sections to see an end to end solution for this which will incorporate the use of System Centre Orchestrator.

How does System Centre Orchestrator play a part in this

Orchestrator is mainly the tool which is now recognized to be the foundation of any new connectors for not just Service Manager but for any System Centre tool and with some respect to any tool.

Integration packs exist for each technology within the System Centre suite and therefore allows us to build an end to end workflow that would carry data from a bespoke tool all the way to the Service Manager CMDB.

Building the connectors - Scenarios to consider

So we will use two scenarios here which will essentially cover everything and will explain the reasonings behind picking those scenarios so you can understand what is required. First we will start with the foundation of each of the design requirements for the connectors themselves within each scenario.

Scenario 1 - Building a connector from scratch

The first scenario is where we have to build everything from scratch where there are no foundations for anything there at all in terms of a bridge which may already exist or any familiarity of how it would collect and store data.

Below are some design sections to consider in this scenario.

Data Points the tool uses

Most tools would tend to use a SQL database to keep all of its data recorded, then again it would be wrong to assume this for every single tool as some may perhaps use an internal database or another based off a different SQL platform such MySQL, NoSQL or even within SDKs such as PowerShell/VB/WMI and even a CSV in some cases.

Attributes of Interest

When it comes to forming the actual connector itself, we also need to think of the aspect of creating the attributes which we are looking to extract from the database so when we do go to populating the CMDB we have the correct information at hand to do the correct CI mapping.

In order for us to do this we need a) the connector but we also need b) a management pack which will contain all of those properties in which all of those attributes of interest would be mapped to.

Say for example we develop a connector for something like Sophos. What would be valid information we would want? For example it would be things like;

Name (Name of the managed device)
Agent Managed (Normally defined by a Boolean data type with 1 being true and 0 being false)
Version (Version of the agent)
Sophos Management Server (Name of the Primary Sophos Server)
Last Scan (Last scan performed)

This would be a good basis to start on to gather the correct information we need.

Correct permissions to access the data

Within the default connectors you are normally prompted to provide a service account which would be used to have enough permissions to not only the tools itself but also the database (if applicable) as well to be able to have enough rights to extract the information.

It is best to have this planned out and implemented before continuing to build the actual connector itself.

Data synchronization scheduling

This can be set however often as you want, however it would be great to understand more of the baseline of the tools which you are using to extract information from to get a better idea of when to actually schedule a synchronization to ensure that you are capturing new or modified data in the most correct way.

For example where Sophos is concerned one of the main functions is how often scans run on each machines as well as if they detect anything which is considered a threat would have an influence on when your schedules should perhaps run. At the same time you may have managed machines in groups where each could be segregated by departments and have different settings for each etc

These are all elements to consider.

Scenario 2 - Enriching a connector which already existsScenario 2 - Enriching a connector which already exists

This is an interesting scenario, and the reason why I've picked this is because there is indeed something I had picked up whilst using the default connectors.

Lets focus on the System Centre Operations Manager CI connector for example. I had touched on this on the previous part. When it comes to the synchronization of data you will notice with everything it gathers what it doesn't actually collect is the health states as well as information of its agent managed status or which management server it would be reporting to.

To me this is quite key information for a CMDB and should be populated.

Data Points the tool uses

We know System Centre Operations Manager utilises two databases being the main operational database and also the Data Warehouse. So we already have a source for which we can obtain information from. On the other hand we also have a PowerShell module in which we could utilise as well, technically this connector is being built from scratch but only really to enhance or cover what the default one is not doing.

Attributes of Interest

In this case we want to capture the attributes which we are missing so it would be properties like;

Name (Name of Managed Device)
Is Available
Health State
Availability Last Modified

Correct permissions to access the data

Ideally utilising the same service account which was used for the original CI connector which may already be in use in Service Manager would be the best route to take in this instance.

Data synchronization scheduling

This should ideally mimic the same schedule as your SCOM CI connector as you will also want to keep up with the health state changes as well as the management information of each agent.

Overview - High Level look of the connector solution

Below is a visio digram which provides an idea of how the custom connector solution end to end would work

Building the connectors

Now we will go into the step by step mechanics of creating the overall connector solution from end to end. We will go with Scenario 2 as it covers in fact both scenarios for the fact that the whole thing has to be created from scratch but at the same time there is already a familiarity of what to actually build from.

Creating the management pack

So the first piece we will create the actual Management Pack itself.

The management pack will contain all of the attributes in which we require and you will see how this will all be linked together once its eventually imported.

Firstly you will need to make sure you have the Service Manager Authoring console installed, which should be bundled with your Service Manager installation media depending on what version you are running.

Open the Service Manager Authoring Tool
Click File - New
Type in a name and location for the Management Pack. In this case we will call it something like SystemCenter.SCOMAgent.ManagementPack so we have a clear understanding of what the MP is for
Once done right click the Classes selection in the explorer on the right and select " Create Configuration Item Class "
Type in a meaningful name with no spaces, something like SCOMAgents then click OK.
Once done you would then go to the Create Property radio button so that we can start to add the attributes we need. So if we take notes of the properties above in Scenario 2, these are what we would need. So type in the Property Name where appropriate and click OK.
Configure the details where necessary, however it is best sometimes to leave its data type as string to avoid any issues. (Note: Preferably a property such as Name would be a key value)
Before you are ready and completed you want to seal the Management Pack so if you have visual studios installed and have the devtools available. You would bring up the Developer command prompt for cmd or Powershell window and run sn.exe -k <Location of the key>.snk . Then right click the Management Pack and select Seal Management Pack and select the key you have just created.

Now that's done you will now want to import the management pack so now we perform the following;

Open the Service Manager Console
On the Administration page right click the Management Packs section and then select import.
Browse to the Management Pack you had just created and click OK.
Once ready click Import and eventually you should see it has successfully imported.

Creating the views in Service Manager

Now we have the management pack created for our CI Mappings, we now need to see this information by creating some views which are specific to the class which we had created for the management pack.

Open the Service Manager Console
Select the Configuration Items tab
Right click the root and select Create New Folder and name it something like SCOM . I tend to use folders to group views of classes so that the CMDB has more of an easy organisation in its structure. Before clicking OK, click the New button so that you can create a New Management Pack which you can store its views. Then click OK.
Right click the folder and select Create View.
Type in a name for the view such as SCOM Agents
In the criteria section select browse, then change the tab from Frequently used basic classes to All basic classes and then search for the class which you had just created and select OK.
In the Display section choose all of the properties you had created in the Management Pack then click OK.

Now you have a view created for your SCOM agents Management Pack. So now whenever your connector does run this view will be populated. So next will be to build the actual connector.

Creating the connector via SC Orchestrator

Now with the management pack created and imported into Service Manager, we now have CI attribute mappings ready to go. So now we then move onto the development of the actual connector itself via System Centre Orcrhestrator which we will not only look at step by step but also look into the mechanism of how we make a successful connection to the database.

The main prerequisites for this would be the following

· Microsoft System Center Orchestrator

· Microsoft System Center Service Manager Integration Pack

· SQL Management Studios + Client SDK + Command Line Tools

The first two are what you will need to formulate the connector logic and the SQL management studios is needed for us to write the PowerShell logic to open a connection to the SCOM Database Server.

So here are the pieces in which you need to create the runbook. First you will need to do the following;

1. Open the runbook designer

2. Right click the Runbooks folder and create a new runbook

3. Make sure the runbook is checked out so you can edit it

4. Use the following boxes below to create the activity steps

So Step 1 is the actual initiator of the runbook which will run at a schedule very similar to how the default connectors would work. The time at which you specify to be started is at your discretion. For this example I will kick this off at 9am which is roughly the time similar to the default SCOM CI Connector within Service Manager.

Step 2 is where the actual data point connections come into play as we will be using a PowerShell script which will be doing the interrogation on the Database Data point level.

You will need to enter values for the following variables

· $Servername

· $Databasename

· $Query – Where the MonitoringClassID part is as it could be different in your environment

· $User

· $Pwd

These can be placed in the variables section.

Below is the PowerShell script which is used to make a connection to SQL. In this case it has been modified to contact the SCOM Database as seen below. Enter this script into the Run .NET Script box and select PowerShell as its source.

$servername = “<Database Server Name>”

$Databasename = “<Database Name>”

$Query = "Use OperationsManager select distinct * from dbo.ManagedEntityGenericView Where FullName LIKE '%HealthServiceWatcher%' AND MonitoringClassId = 'A4899740-EF2F-1541-6C1D-51D34B739492'"

$user = "<Domain\Username>”
$pwd = "<Password>"

$QueryTimeout = 120

$Connectiontimeout = 30

$Connection = New-Object System.Data.SqlClient.SqlConnection

$Connectionstring = "Server=$servername;uid=$user;pwd=$pwd;Database=$database;Integrated Security=True"

$connection.ConnectionString = $Connectionstring

$connection.open()

$command = $connection.CreateCommand()

$Command.CommandText = $Query

$result = $command.ExecuteReader()

$Agents = New-Object "System.Data.DataTable"

$Agents.Load($result)

$Name = $Agents | ForEach-Object {$_.DisplayName}

$HealthState = $Agents | ForEach-Object {$_.HealthState}

$IsAvailable = $Agents | ForEach-Object {$_.IsAvailable}

$AvailabilityLastModified = $Agents | ForEach-Object {$_.AvailabilityLastModified}

$BaseManagedEntityId = $Agents | ForEach-Object {$_.BaseManagedEntityId}

Once created then go into the published data option and create the variables which you see in the script which are

· Name

· HealthState

· IsAvailable

· AvailabilityLastModified

This will allow you to pass the variable data across to the other activities.

Step 3 is where all of the checking of each CI happens. Once all of the CIs have been discovered from the second step, it will then check it against the Service Manager CMDB to see if this record exists or not. Below steps contain which will be applicable if it exists or doesn't exist.

Step 4 is used to deal with CI s which have not been found in the CMDB as already existing. So this step will then create the CI with all of the attributes which we had designed to be placed into the CMDB.

Note: The link condition mentioned above using ^$ is a null pattern

Step 5 deals with CIs in the category of them already existing within the CMDB. So this step will then detect which attributes are different from which has already been discovered and will update them appropriately thus keeping data integrity at a solid point.

Note: The link condition mentioned above using ^$ is a null pattern

So your Orchestrator runbook should look something like this.

Once the runbook has been saved and checked in you are now ready to run.

When successfully ran, if we go back into the Service Manager Console we should now start to see CI's populated like below.

Overview of series

We have now investigated the whole spectrum of how we can not only have a fully automated CMDB from end to end with all of our toolsets, but also how to harden and standardise the toolsets to where full data integrity is maintained throughout the whole process.

When carried out correctly achieving a 99% success rate is not out of reach, course success rate and the data which is being corrected depends on the up keep on the overall toolsets from the backbone of data points to the GUIs which connect into them.

Building the perfect CMDB with System Centre Part 3

Sat, 12 Dec 2020 12:47:07 GMT

Low Level Design - Enrichment of Default Toolsets

Introduction of part 3

So this next part will focus on the enrichment of the toolsets which will allow you to obtain the best information and the cleanest in terms of integrity when it comes to your CMDB.

The default toolsets being focused around;

System Centre Configuration Manager
System Centre Operations Manager
Active Directory

Enrichment of Active Directory

The backbone of your infrastructure most certainly has to be clean as it being the foundation for your orphaned objects if you are not careful of its up keep so will go through a few scenarios to consider in more finer detail.

Temporary Build Platform

Though this section does have an SCCM/MEMCM influence, we have to remember that computer objects at the end are always going to end up here. Depending on your environment it may go directory to the "Computers" OU or it will go perhaps to a designated OU of your choosing whether this is done by a selection within the AD tree or a variable to decide.

We find that we have template builds which can cover various technologies such as building tools (MDT, SCCM) but also templates which derive from hypervisor platforms (VMWare, Hyper-V, SCVMM). These can create AD objects but we also have a process in which we may rebuild multiple machines which either use the same computer object or perhaps a different name afterwards leaving the older AD object orphaned.

Be very mindful on the up keep of AD. Though these can be excluded through various options which we will cover later, they still need to be removed from AD at some point or another.

Organisation of OUs & Group Memberships

A child section perhaps of the section above. Once new AD objects come in they would need a clean up to placed into the correct corresponding OUs which apply to the AD Objects whether they are computer or user objects.

When it comes to finer discovery methods which can come from many tools, some organisations may find that just having a discovery across the entire domain or forest to be more intrusive and discover objects which they don't want to be found, or in the case of which specific objects they would expect to be in certain OUs, simply are not.

This does sound like a very minor action, but it really does make all of the difference when it comes to the overall integrity of the CMDB in general. Because anything not covered here, will eventually deteriorate once it reaches up to the CMDB level where you would have to develop more filtering enrichment than necessary to correct these things.

Attribute Population

Perhaps the most overlooked and maybe the most important of all in fact!

When we think of the attributes in AD you may think of base ones such as

Name
Description
OS
Pre Windows 2000 (Netbios Name)

But we have to remember there are so many others which are not filled in and they are so key to the enrichment of your CMDB as they are essentially the CI's that are going to be synchronized. Here are a few examples;

Department
Manager
Owner
Office
City
Address

The more information in your AD Objects the better information you will be able to extract overall from your CMDB. You will find once it does reach to the CMDB level these values would be empty. And as you don't have an asynchronous level between your tools and the CMDB the gap becomes bigger in terms of maintaining integrity on both sides.

Low Level Design Diagram of your enriched AD

Below is an idea on how your AD would look when enriched. So you would have your specific object classes which would all have fully or relevant populated attributes which would then feed into your CMDB

Enrichment of MEMCM

Your hardware and software inventory information all resides here. Also not to mention its also another pit stop for which your Active Directory will know doubt connect to, so its also a very important piece to have refined.

Hardware/Software Inventory Scheduling

This is a tricky one, as we can't always account for how many devices within MEMCM will actually be online as such but good scheduling is also very important when it comes to these particular scans.

Another key element of the CMDB information is the Hardware and Software information which relates to each CI that gets pulled into it, so it is absolutely worth investigating your client settings (whether you use default or a custom device one) that you have the schedules done at one which is realistic as well as balances out the times in which it can get accurate scans.

By default these are set 7 days, most recommended or should I say popular tend to be between 1-3 days with 1 day perhaps being too frequent whilst 2 days gives a more appropriate amount of time to catch others which may have been off the network.

It would be a recommended idea to have the scans and the connectors for MEMCM to be somewhat in line with each other to ensure you are pulling a greater amount of successfully scanned machines to ensure your data is definitely up-to-date.

Temporary Build Platform Continues

With orphaned objects of course being in AD, there is also the bigger one of them being in MEMCM which can not only destroy the integrity of the CMDB but can eventually cause issues within MEMCM as well.

Repeated builds on objects in MEMCM can indeed create duplicates, especially when identical information is shared on both records such as Mac Addresses and SMBIOSGUIDs. But this can also be caused if you have an object in AD which may not necessarily be active and this gets pulled in from your discovery methods and can leave greyed out records and can cause ambiguity when managing devices for other reasons.

There are a couple of ways in which you can avoid this. First lets look at how we bring objects into SCCM via the discovery methods. If we look at the settings below.

Here we can zone more into specifics of which OUs we want to bring in which is a filter on areas we dont want to include. But then we also have the additions of more advanced filtering which allows us to bring in objects which have either been active or not had password issues within a certain amount of time. Though we would expect to have a lean Active Directory, we still have to factor in the real-life scenarios of that there could still be some objects which would be troublesome.

Once that is sorted, how do we maintain the same cleanliness in MEMCM. We would look more into the Site maintenance tasks which would focus on clearing orphaned objects.

Another point to factor in is how we build machines in MEMCM if using All Unknown Computers. It is indeed convenient but it can sometimes make issues like these more tricky, where as pre-staging machines and with appropriate collection designations can make this a lot easier to manage otherwise.

Collection Management

Important to have very precise and to the point collections, ones that would be able to identify what the collection is for and the type of members which are in the collections.

Where Service Managers CMDB is concerned it focuses on the devices by collections, and if we are using a strategy in which the collections are query based and are specific to group membership or particular OU membership this can indeed help in terms of the enrichment of your CMDB structure.

You have the option of wanting to bring in devices using default collections such as;

All Systems
All Users
All Desktop and Server Clients

But you may want to exclude certain objects rather than bringing everything which might not necessarily be required if it was something like;

Test Accounts
Service Accounts (Optional)
Known Temporary machines/users

Multi Platform Management

For those who perhaps manage devices in MEMCM which are other than WIndows such as;

Linux/Unix
Mac OS

We have to remember that there is no actual inventory that gets migrated when it comes to Linux/Unix when moving into your CMDB so you will have to be very mindful of this. Mac OS I believe may suffer the same fate, so some custom work would have to be done in order for you to bring these devices into your CMDB where inventory is concerned.

This will be covered more in the next part of this series, but we do have a development which can import this for you which can be found here https://gallery.technet.microsoft.com/Linux-Software-Inventory-14402621

Low Level Design Diagram of your enriched MEMCM

So we can see the devices and users organised within the collection containers. We can also see the hardware and software inventory which would run on schedule for those devices and they would in turn be pulled into the CMDB from the SCCM connector within Service Manager.

Enrichment of SCOM

This is where things are going to get very interesting here! This tool has such as wallop of importance to your CMDB. As this is where you will obtain all of your attributes for every single kind of class which your CI fits. But to get it there takes a substantial amount of work, but we will explain how to enrich this side.

Management Pack Referencing

The Management Packs in SCOM are truly king in this scenario, especially the ones which are marked Discovery and Library. Most management packs contain this kind of structure where the Library Management Packs contain the classes of all of the objects which make up the overall technology, and then the discovery management pack is then used to discover those attributes on the machines which have a SCOM agent installed.

By default Service Manager will have some base ones it will start to pull in from, but in order for you to truly unlock its power you have to import the rest of the management packs you have in order to really obtain all of the attributes for all of your classes.

Versions are also quite important, as any old versions can complicate the type of references you would require in order to get your objects discovered correctly. Save all management packs that you download either in a repository or at least make sure that they are within the default C:\Program Files (x86)\System Center Operations Manager Management Packs folder.

I wouldn't say this section has much of a focus on an enrichment perspective, but more of a note to say ensure your management packs are up to date and you have the original files for them.

No Management Pack for SCOM Objects??

Sadly enough when it comes to the connections from SCOM to your CMDB, there are no default attributes pulled in which actually indicate if a server is indeed managed by SCOM or not as well as its current health state.

Luckily this is covered in the next section of this series.

Configuration Churns

This section requires the most enrichment in every sense, though this is slightly off topic as this is more of the SCOM alerting connector for those who are using Service Manager as the basis to route alerts into.

Any configuration churns which impact very negatively on your SCOM estate such as;

Thousands of alerts
Performance Counters high
Too many state changes
Agents health and connectivity going up and down

These can cause massive amounts of traffic where alerting is concerned. And when this happens if you create a connection from SCOM to Service Manager you will potentially flood everything with nothing but incidents raised with a load of alerts which may not represent what your estates state really is.

Low Level Design Diagram of your enriched MEMCMLow Level Design Diagram of your enriched MEMCM

So the SCOM Agents and classes all relate to the management packs which perform all of the discovery methods and they then feed into the Service Manager connector. So having the correct management packs imported into Service Manager is very important.

Coming up on the next part

The next part will go into detail on how we create the custom connections to all of our other toolsets as well as developing a way to capture all of the attributes which relate to those same toolsets.

Building the perfect CMDB with System Centre Part 2

Fri, 27 Nov 2020 15:05:59 GMT

High Level Design Look - Painting the picture of how it would look on various levels

Recap of previous part

To see the first part of this series you can view it right here

We spoke about how to go through the analysis of all of the toolsets in which we have within our estate which we are able to either pull CMDB data from and for some which may perhaps need an alternative way to connect to it to pull data from which can range from bespoke tools all the way to legacy based systems.

Introduction to Part 2

This part will now cover more of the HLD (high level design) of how the overall CMDB solution would look which we will have multiple scenarios all accounted for. Also we will dive a little into the integrations with other toolsets which also carry a CMDB role to them in where perhaps a synchronous or asynchronous relationships can be made to each other.

These should hopefully cover most kind of setups and environments in which we allow room for scaling up the reach of the CMDB connections across all toolsets which maybe available.

Scenario 1 - Design of CMDB with default connectors

Here we have which would be an out of box setup which you would be able to utilise from System Centre Service Manager which already contains the following connectors by default being;

Active Directory
Microsoft Endpoint Manager Configuration Manager
System Centre Operations Manager (CI and Alert Connectors respectively)

In the diagram each area is tiered with information aligning next to them to give the idea of the entire workflow on how data is passed through from the toolsets all the way up to the CMDB.

This would be considered a base CMDB setup which should be able to account for everything such as all computer objects as well as hardware/software inventory on the perspective that all CIs are indeed managed and maintained across the main System Centre products as well as the backbone of your infrastructure being Active Directory.

The connectors for both the SCCM/MEMCM and also the SCOM all connect to the database servers which hold the operational databases for each tool respectively which then feed into the connectors then populates the Service Manager with the CI information which in turn populates the CMDB.

Active Directory on the other hand would connect through LDAP to the domain/forest which you choose to pull from which feeds into its base connector within Service Manager.

Scenario 2 - Design of CMDB with connections to all toolsets

In this scenario is where we promote more of the scalability within our CMDB solution, where we are now able to broaden the scope to every single toolset that we have on a global scope.

Very similar structure to the previous Scenario however we have now introduced another piece to the solution which will help with the scalability of reaching the toolsets.

If we look at Tier 5 there is an object which would represent all "Bespoke Tools" which is my way of perhaps grouping not only solutions which may be developed in house, but there may also be Microsoft based Products in which don't perhaps have a default connector such as Exchange (there is one for SMTP channels), Sharepoint and other various products.

So when we look at the next tier up we see a picture of multiple products, these represent a good example of the type of methods which would be used to pull data into Service Manager such as;

Excel - .CSV, XLSX
Access - Access Database Files
PowerShell - Connection via PowerShell Scripting + SDK integration
VBScript - Utilising a VBScript for perhaps more legacy based toolsets
Python - More programming scripting which maybe able to interrogate much further as well as another alternative
SQL Database - Though this is used as a somewhat default for the connectors which already come with Service Manager, other toolsets wouldn't have a direct plugin so a custom SQL query or ODBC connection maybe required for this.

How we would connect with these methods? This is where the tier 2 comes into place where we would be using System Centre Orchestrator which was also meant to be a solution for which would be designed to also provide custom connectors for System Centre also. We will investigate more into this side once we get into Part 3 of the series

Scenario 3 - Design of CMDB with all toolsets with Integration to another CMDB

Not necessarily a popular scenario or setup as such but is indeed a means to keep an automated CMDB in one space which can then synchronize to another.

Again this scenario is very similar to the 2nd however we are detailing a high level design of a fully automated solution of the CMDB which may not be the primary technology which is used for all Service Management as many organisations may use a technology such as ServiceNow. And with the power of ServiceNow you may not necessarily have a connection point from many products to go direct into ServiceNow very easily, though there are products such as SCCM/MEMCM which can have a connector to ServiceNow as well as SCOM with tools around Event Management and even others such as Evanios.

This scenario provides an idea of how it can be done, with that being said this would be more off scope as to where the series will be focusing more building the actual workflow within System Centre primarily. But at least with this series once your CMDB automation solution is built you can indeed go that route to expand across. Kelverion Integration packs are one of a few solutions which can provide a connection between both for example.

Overview of all Scenarios - Target Scenario

Overall we have analysed the tools in which we have, and now we have laid out a high level design on what we feel that design will look like. Scenario 1 is ideally where we want to start as a foundation but we know in a real world scenario other tools would fall by the waist side with this setup so we want to allow room to scale to capture everything in Scenario 2.

With all the toolsets we have I would summarise it as with the tools we dont have connectors for we simply create a custom one to bring them altogether. Whilst with the others which may already have a default connector we simply need to enrich and prepare for quality control before and after setting up successful connector synchronization.

Next on Part 3

Part 3 will then focus on more of the low level design of how we establish the connectors to products which already contain one, and will also look at all of the best practices and ways on how to enrich and place a quality control for the integrity of everything being synchronized into Service Manager so we don't create an issue of out of date or orphaned CI records.

Building the perfect CMDB with System Centre Part 1

Sun, 15 Nov 2020 12:17:52 GMT

Analysis of the tools in your environment

Introduction

This will be a multi part series on investigating on how we can scale our System Centre toolsets to integrate them altogether to build and develop a powerhouse for having the perfect CMDB.

There tends to be a lot of challenges in terms of not only maintaining at least a 99.99% accurate CMDB, but then the bigger challenge is then to have one which is actually automated. From various projects and customers I have dealt with over the years a lot of the processes have been mainly manual, or the most I've seen will go up as far as semi-automated where you may have some tools automating some of the work whilst others are done manually.

Solutions such as ServiceNow are great, and even with the integration with SCCM to populate its CMDB is also quite handy, but the caveat is that it's truly dependent on a constantly healthy SCCM where there are objects which are not orphaned or out of date. Not to mention that SCCM may not be the sole source of all other toolsets which may reside in your environment so there is still a semi-automated approach to the method of trying to keep your CMDB always up to date.

This series will help to shape and understand the strategies which can be used with System Centre to provide a fully automated approach to keeping a clean, healthy and accurate CMDB. In System Centre the main technology to represent the the CMDB infrastructure is the SCSM (System Centre Service Manager) technology which will be the heart of the solution.

Purpose for the series

Designing, planning, building as well as maintaining an accurate CMDB is not an easy project by any means. And because of the complexity of it is the main reason as to why the approaches have always been more manual/semi-automated. Though manual maintenance approach can be more accurate out of the two, it can still leave space for errors as the environment would be changing at a pace in which user interaction cannot keep up with even if you were to work on it on a monthly, weekly or even daily basis.

This series will explain in detail over several parts how a CMDB can be fully automated with the right planning and right strategies.

What this part will focus on

Part 1 will be to focus on the analysis of the toolsets in which we have to understand how they can all be brought together into one centralized approach and prepped for automation. The most important one of all I would say, in order to capture a discovery phase of what is actually out there and how we will work to bring everything together.

Planning

Analysis of Toolsets

If we take a look at a technology such as SCSM (System Centre Service Manager) which has its own CMDB function, we see that SCSM already contains its own connectors such as;

Active Directory
SCCM (System Centre Configuration Manager)
SCOM (System Centre Operations Manager)
SCVMM (System Centre Virtual Machine Manager)
Orchestrator (SCORCH)

These standard connectors are a great foundational base to capture CI's (Configuration Items) within the environment. However it may not account for every single real-case scenario in which other tools exist that there are no actual connectors to go by.

Others could exist being for example;

Anti Virus Solutions i.e. Sophos, Symantec, Standalone versions of Endpoint Protection/Defender
VMWare
Exchange
Bespoke Toolsets

These all play a part of information for each CI which would get left out of the equation.

Also to expand on some of the other toolsets such as the ones below, so that we understand not only just the work of connecting them but also to ensure that the integrity of the data is correct and certain risks need to be highlighted as well as addressed.

SCCM

Your SCCM may contain multiple environments that could be split into multi environments, or may not have a lot of maintenance or up keep, so even though SCCM maybe a main point to source most information for your CIs, it does hold a single point of failure to some degree.

Also to take into account of the type of devices in which SCCM can manage being primarily Windows but also Linux/UNIX. Now with LINUX/UNIX machines we have to remember that hardware/software inventory doesn't not only get show in a resource explorer within SCCM but also it doesn't get imported into SCSM by default as well, that's also assuming your Linux/UNIX estate is managed by SCCM of course.

Apple Devices could also be managed by SCCM, but how about if you are using a tool such as Casper? That's then another tool you have to worry about in order to obtain CI information from.

SCOM

SCOM essentially has two connectors, one for alerting and another CMDB information.

Where the alerting is concerned, a point of concern would be of course the health state of SCOM within itself. Especially configuration churns where non tailored or tuned environments can end up transferring a configuration churn to the CMDB which could cause catastrophic issues. There are other tools which can pull in raw alert data such as Event Management based modules and other tools such as Evanios, but again it's another semi-automated approach which we are looking to move away from.

Another challenge is then the CMDB connector it has, now in order to get all of the information from SCOM into SCSM we need to add every single dependent management pack to reference from which exist in SCOM. Mostly these consist of library and discovery management packs to understand the various classes.

SCVMM

The connector in SCSM for this is not what you think. This connector would import information such as VM templates and other objects which aren't necessarily CI information around VMs, hosts, datastores etc..

In order to obtain this information this comes primarily from the information within the SCOM CMDB connector and at that we still require the management packs for SCVMM to be present as well as various dependencies.

Active Directory

You might be wondering why I've actually placed this one last? Well active directory is not only just the backbone of an infrastructure but it most certainly is for all the tools mentioned above.

This is why its imperative to ensure the integrity of data is kept right because inaccurate data in Active Directory will essentially spread into tools such as SCCM/SCOM and then eventually brought into SCSM.

Now the tools above do have their own sources of filtering data in Active Directory but this can really only go so far. Also not to mention a scenario in which we have build rooms or test environments in which objects are constantly being rebuilt, deleted or templated can also have a long lasting effect. So not only designing a strategy to ensure data integrity but to also account for and even label the objects which reside in this space is just as important perhaps more.

Legacy Toolsets

Whilst I imagine most organisations maybe trying to use the latest and greatest of technologies available, unfortunately this may not always be the case. Most of the time there will be some legacy tools around which are hanging by a thread but still very business critical to manage other legacy systems.

They can be from anything as old versions of SCCM (SMS 2000,2003, SCCM 2007 even SCCM 2012 would be considered legacy now) as well as old versions of SCOM (MOM 2005, SCOM 2007 etc) we have to account for everything which is being utilised within the business.

The challenges we would also face is not just the time span of the legacy estate but also the technology gap between them which may make it more challenging to integrate or to pull data from, and that's just from a manual perspective let alone on an automation perspective.

Where the toolsets centralise data

Most tools these days utilise a SQL Database or some form of database (MySQL, NoSQL, OracleDB)

But then some tools can be more different and perhaps keep there information in a file within an installation directory, or perhaps just within the realms of PowerShell in their SDK.

Legacy tools may use some of those mentioned above but maybe more in the realms of having to use VBScript or even something more universally legacy than that.

Overall we need to capture of the different avenues in which a connection of possible integration can be reached by investigating each point where communication can be reached.

Possibility Of Bi-Directional Synchronization

Perhaps one of the most debated decisions when it comes to the synchronization of CI objects.

In terms of the discovery phase this would seriously depend on the role in which the CMDB will play and how it will fit in your environment especially where a user operations team would fit and other various tier based technical teams.

A one way synchronization I can see the benefits as it allows for some user based validation and integrity checking, where as a bi-directional method would of course supported the fully automated approach in which we are discussing but then we may not be able to fully trace the historical side to see if that decision of synchronization was indeed accurate or not.

If we look into a technology such as SCOM, now when it comes to monitoring it uses objects which have the same kind of structure as the scenario of one way or bi-directional. Monitors in SCOM mostly hold a bi-directional configuration so an alert can be created for lets say a spiked CPU performance, and before you could even get to the ticket or alert it may already be auto-resolved. Whilst rules and a manual reset monitor however take an approach of non auto resolution so just really one way. I guess one way you can look at it is full automation is convenient whilst a manual/semi-automated has more chance of a security footprint and to obtain a baseline of how and why.

With the fully automated approach we will be expanding on maintaining a historical baseline and security footprint as well in which we will expand on within the next parts.

Next up on Part 2

Part 2 will contain all of the technical side and breakdown on how to design a fitting solution. A high level diagram will also be shown to show all of the integration points and how they will be tackled to bring everything all together

New book currently in the works

Sun, 15 Nov 2020 10:53:46 GMT

Not for a little while i might add!

Firstly thanks again to all of those who have purchased a copy of my self titled book "Self-Motivation: Anyone Can Do It, If I did it" which is still available within outlets such as

Will also have 10 signed copies available which you can get in touch with me for. But I'm announcing that a new book is currently in the works, which will take some time to write.

What will be the name of the book

The title of the book itself I already have, but will reveal the title in due time whilst I build the planning around the book.

What will the book be about?

For those who have read/purchased the first one it was tailored to more of a biography of how my career started up until a certain pivotal moment, and was utilised for more of a motivational type of book to not only promote the prosperity of a career within the IT industry but also any type of goal or mission in general, and one which I was more than happy to share my experiences with.

This particular book is sort of a continuation but not in a biographical form, more of a tricks of the trade/tips/advice and knowledge from everything I've learned broken down into several chapters.

The idea of the book is to shape and mould someone to be a strong entity in whichever path they choose to take, and though my career is primarily IT there are other ventures and avenues I've done well in so it will be quite a universal type of appeal to the book.

When will the book be released

Difficult to say at the moment but I would put a marker between end of 2021/beginning of 2022.

I purposely want to take my time in creating the next one so the breadth of knowledge is absorbed in finer and greater detail that will pull from real life scenarios and experiences. Most will be from my own some may be from others.

SCOM - Management Pack Creator Official Version Released!

Wed, 09 Sep 2020 12:51:59 GMT

Build management packs in a matter of seconds!

Introduction

Back in 2015 i was working on various custom management packs which required a lot of bespoke settings and configurations which were required amongst various projects and took it perhaps as a challenge to see if there was a way of speeding up the process. Not necessarily just about the speed of the production, but also the overall quality of the produced management pack.

We find a lot of SCOM administrators and management pack developers tend to refer to our own created management packs and tailor around those which is great and in fact i do so myself, but what if you are in a situation in which you need to create something or add in addition in something you may not already had in your management pack to begin with?

Here is where i had developed the earlier versions of the POC called the SCOM Management Pack Creator.

A demo of this solution can be seen here https://scomathon.com/coffee-break/custom-mp-creation-with-powershell

Previous Releases

There have been various releases of this such as the ones below;

SCOM Management Pack Creator V1.0 - https://www.walshamsolutions.com/scom-scom-management-pack-creation-script-v1-0
SCOM Management Pack Creator V2.0 - https://www.walshamsolutions.com/scom-management-pack-creator-2-0-officially-released
SCOM Management Pack Creator - PS Module Version - https://www.walshamsolutions.com/scom-management-pack-creator-ps-module-officially-released

Each of these had quite interesting in enhancements from growing from a wizard type of form into a full blown PowerShell Module in which you can import to run the functions manually. Now with greater enhancements and creating a bug/development list (see below) we have now been able to consolidate all the above into a bigger beast to present the official version.

Also have been fortunate to achieve over 2000+ downloads combined on these previous releases!

This Solution vs Management Pack Fragments

Management Pack fragments are brilliant ways in which you can take the XML code of a specific fragment which has been developed before in which the administrator can then replace with their own IDs & properties in which they need, and is also a great way of being able to then hold onto them when they become your next successful management pack which you can then model from.

Where the solution here is different is that what you find with fragments and your own management pack fragments, you are required to search and replace multiple IDs and this can be lengthy, especially when you have a lot of fragments, classes, IDs, references etc

Not to mention they are also case sensitive so that adds another level of complexity. Also bear in mind that base classes can be completely different which will have a knock on affect to class fragments and anywhere else which is referenced. So though this method is great to use you still need to have a fair on idea on how you will develop the management pack.

The solution i have here will not only create the MPX fragment but it will also use your IDs and base classes and build them straight away in a matter of seconds, a very similar mechanism you would see if you were to create a SCOM management pack from the Operations Manager console!

What Can you create with the Management Pack Creator?

See below all of the components you can create for the Management Pack;

Create Classes - Create brand new classes which can use base classes or custom based classes depending on your management pack requirements.
Create Properties for Classes - Create properties for your classes which attributes can be populated in conjunction with your discoveries. Ability to decide which ones will be a key property for your discovered instances.
Create Run As Profiles - Create Run As Profiles which can be assigned to your discoveries, monitors or rules in which require accounts with the appropriate permissions to carry out monitoring actions for your classes.
Create Relationships - When relating other classes to each other if using them as references for other fragments of your management pack, you can create a relationship fragment to join both together.
Create Discoveries - Create various discovery methods for your classes. You can choose from PowerShell, WMI, VBScript or Registry. You can also create discoveries for ComputerGroups and InstanceGroups.
Create Monitors - The ability to create monitors around Events, Performance, Services, Services in conjunction with performances and many more.
Create Rules - The ability to create rules around Events, Performance and rules which use a PowerShell Script.
Create Views - You can now create every view you need from State, Alerts, Performance, Overrides, URL and Events views.
Create Folders - Create a folder for your management pack as well as subfolders in which you can nest underneath the parent folder and also place your views into.
Create PowerShell based Monitor - The ability to create a custom module which is based on using PowerShell where you can create a monitor with any condition you choose based on the PowerShell script you write.
Create Diagnostic Tasks - Create diagnostic tasks for your monitors to run under a specific severity to analyse more information on the health issue.
Create Recovery Tasks - Create recovery tasks for your monitors to run under a specific severity to help automatically resolve the issue with your monitored instance.
Create Console & Agent Tasks - Create Console & Agent tasks which are applicable only to your monitored instances relating to the class which you have created in your management pack.
Create Knowledge Articles for each components - You now have the ability to add knowledge articles to your fragments whether that be discoveries, monitors, rules etc

New Features

During the transition to the official release of this solution, new features and additions have been made along the way.

Below are a handful of brand new features to help enhance this solution.

Create-PowerShellScriptDiscovery Function - A new function has been added which allows you to create an all purpose PowerShell Script best fitted for an actual discovery.
Add additional Registry discoveries - A new function has been added called Add-SCOMMPRegistryKey which allows easier creation of the Registry discovery method to attach each property to an appropriate setting and mapping,
Add Function for WMI Discovery for adding Properties - A new function has been added called Add-SCOMMPWMIProperty which allows easier creation of the Registry discovery method to attach each property to an appropriate setting and mapping,
Add Linux/UNIX support - There is basic support to create monitors, rules, discoveries and also agent tasks for your UNIX/Linux estate. Note you will need the Unix Authoring MP which the link is available in the downloadable instruction guide.

Power of the Get-SCOMClassID function

I have spoken of this in the past when i released the 3.0 version but have to re-iterate its powers!

The power of this function is with in conjunction to completing management pack builds on different functions which accept a pipeline and also accept a parameter with pipeline property name.

For example this command has switches to obtain the following;

File which contains the Classes
File which contains the folder classes
File which contains the Monitors and Rules

Once entered, this command can dynamically see every single class in any file which you choose which is great if you dont want to investigate a management pack to find what name you gave the class as this can dynamically display the list and select for you. Then add to part of the command and use to pipeline to another command.

Once you have worked on the intellisense you can then pipeline into a command like specified below.

Note that this doesn't work with variables this will only work when you specify the entire path (quotes, no quotes or spaces) then the intellisense will kick in.

Bugs Fixed

With the previous versions being very well received there have been many bugs which have been meaning to be fixed and rectified in order to provide a smoother and cleaner management pack building process.

Though this solution is used to provide flexibility as well as provide quality of building a custom management pack with speed, certain bugs can impact the speed and quality of the product as well as spending unnecessary time in trying to troubleshoot and look for reasons why your management pack cannot be built correctly.

Or even worse when imported into SCOM to find certain issues in various parts of the modules. Below is a list of bugs which have been corrected.

Get-SCOMClassID now takes file paths with quotes or spaces
Properties in Discovery should have new lines rather than same line
No longer Depends on MPMonitorRuleFile if wanting to use Get-SCOMClassID when in junction with creating views
MPElement is missing from PowerShell Scripts now fixed
Quotation marks on PowerShell scripts need to start from MPElement and end at first $
Change SeverityList and Severity switches in views as comments to avoid setting wrong settings
Change MaintenanceMode switches in views as comments to avoid setting wrong settings
Registry Discovery Datasource uses Windows.ComputerRole instead of Windows.Computer
Modify Add-SCOMMPRegistry function where if PathType is 0 then AttributeType cannot be used
MPElement missing from additonal registry keys added
Computer Role user in Datasource for Registry should just be computer
HKLM needs to be removed from paths
Param line on PowerShell script for custom module needs to be removed
ComputerName parameter should be removed from Performance Monitor function

Where to Download

The Management Pack Creator solution can be found in the following link below.

https://irp-cdn.multiscreensite.com/c1f89e50/files/uploaded/SCOM%20Management%20Pack%20Creator%20-%2...

Instruction Guide

You can also download the instruction guide which also gives full reference to every function in which the module has in the same link below
https://irp-cdn.multiscreensite.com/c1f89e50/files/uploaded/SCOM%20Management%20Pack%20Creator%20-%2... .

How to run the solution

The solution is of a PowerShell Module which can be imported.

In order for this to be working perform the following;

Download the solution
Unzip to a location which you have enough administrative rights
Open either PowerShell or PowerShell ISE depending on which one you will be using to build your management pack.
Run " Import-Module .\SCOMManagementPackCreatorModule.psm1 -Verbose "

How to Create Management Pack Project in Visual Studios

You will require two pieces of software before you can do this;

Visual Studios (Currently Using 2019)
VSAE Authoring Extensions - Latest download here - https://www.microsoft.com/en-gb/download/details.aspx?id=30169

To create a Management Pack project perform the following

Open Visual Studios
Click Create a new project
Type Management Pack in the search engine to find the Operations Manager listings. Select the version which is applicable to your environment.
Enter a name for your Management Pack and then click create

Once created you should then end up with the screen below

How to Build a Management Pack

Once the module is imported you can then use the other PowerShell Script which comes along with it which is the SCOM Management Pack form.

It contains all of the functions needed and required to build your SCOM management pack in which you can then replace the values on the switches to build the management pack you require.

Below is a screenshot of what the form looks like.

The first values you see are variables which create all of the fragment MPX files required for your management pack. As you can see in the example there are already two variables filled out for the Management Pack Name file and Class file.

Below are the lines for every single function required to build out your entire management whether you intend to use all of them or most of them you can look through them.

There are values pre-filled to give you an example of what to put down to create your fragments.

Few things you should know about this process for full completeness of understanding;

The switch that asks for Name like " ClassName " can be written as a friendly name with spaces as the solution creates the ID with the same answer but with dots replacing the spaces. For example i used " PowerShell Agent " as my class name. But the ID will be formatted as " PowerShell.Agent "
Any lines which use " Get-SCOMClassID " enter a full file path for your class file, then delete the prefilled value on the " SourceClassID " or " TargetClassID " switches and the intellisense will work to pick up your IDs.

If there are any lines, functions you don't require for your management pack then simply put a "#" next to the line so it doesn't get action-ed during the script.

Once you have configured all the properties and configuration you require you can either execute the entire script, or you can stagger it line by line if you are looking to see the process develop piece where you can observe changes in the MPX files etc

From here you can now update the script with the location of your Management Pack project which normally resides in the source\repo location of your user profile i.e. C:\Users\Administrator\Source\Repo\Test management Pack

When you run the script you should then see MPX files get created in these locations as seen below.

You can then right click the project folder in Visual Studios and then select to import and then select the MPX file so that it is now part of your Management Pack Project.

Whether you run the script before or after this you should see XML code populated, if running after you will see the XML code appear in real-time as seen below.

When finished with the management pack select the Build - Build Solution option in the toolbar above.

Sealing your Management Pack

Another cool thing we all want to do is to be able to seal our management packs so they end up as a .mp extension as opposed to an unsealed .xml extension.

To do this you need to open up the developer command prompt for visual studios which should be located in your Visual Studios folder in the start menu or installation folder.

Then run the following command

sn.exe -k C:\Key\MPKey.snk

Once created you can then go to the properties of your management pack project and select the build tab and where it displays a box for your encryption key point to the location where you created the key earlier and select it.

Now when you build your management pack it should now create a .mp management pack in the Bin\Debug folder of your project repository.

Troubleshooting

Errors Building due to references

The solution will have disclaimers when building specific fragments to warn you of any references which you may require.

But just in case we will cover it in this section.

Say for example you want to create an InstanceGroup within your class, if you try to build you may perhaps experience an error such as this.

If you see this, it's due to a missing reference management pack which is required. Luckily when you install the VSAE authoring extensions it creates a folder which contains all of the management packs which you may need for this and has them categorised depending on the version of SCOM that you are using.

The best way around this is to perform the following;

Right click your Visual Studios Project References folder underneath your project name
Select Add Reference.
Go to C:\Program Files (x86)\System Center Visual Studio Authoring Extensions\References\OM2019
Select the Management Pack reference you need (in this case its the Microsoft.SystemCenter.InstanceGroup.Library.mp)
Once this is imported click on the Management pack.
Observe the field which says Alias, as this has to match the first part of your base class that is being used on your InstanceGroup Class i.e. The Alias is MSIL, and the Base Class is using MSIL!

Screenshots below will help to explain it.

Intellisense not Working on the Get-SCOMClassID

You will need to specify the entire path so that when using switches such as

MPClassFile
MPFolderFile
MPMonitorRuleFile

Are then taken with respect to analyse all of the classes which reside in each fragment.

From there when you use

SourceClassID
TargetClassID

It should then pop up with all of the classes.

MEMCM Device Dashboard

Fri, 31 Jul 2020 09:06:53 GMT

Ready for consultation for implementation through Power BI service!

Introduction

With the long waited and anticipated revealing, Walsham Solutions LTD now introduces a brand new dashboard which is engineered specifically for MEMCM and is developed with a dashboard available on the Power BI service.

This overall solution contains not only a live dashboard which displays a phenomenal amount of information to paint the overall summary of your entire MEMCM estate, but also additional highly developed pages which correspond to each summarized widget to expand further on CMDB information.

Aim of The Dashboard

Through many years of developing solutions across various clients and organisations with different requirements across the spectrum when it comes to reporting and any kind of report analysis, it's very difficult to standardize in such a way in which a report/dashboard solution can be applicable to all. And this can range from SLAs, SLOs to very specific drilldowns in which MEMCM may contain but require additional work through an advanced SQL/WQL, PowerShell SDK or even having to extend classes to obtain.

Whilst various dashboards which Walsham Solutions have developed have been more related to specific groupings of priority such as the Software Update Dashboard & the Deployment Dashboard. They are very advanced and standardized dashboards but they are more locked specifically to focus on collectives of assets.

This dashboard here is specific and developed to focus more on the individual asset as opposed to the collections perspective. One thing that always remains very true in terms of report detail is to know the full asset inventory detail or a specific detail to just one machine .

This can range from anything from a "Last Logged On User" or a "Last Hardware Inventory Scan" or even to know more intricate details as to its connections and where its connected to in terms of Intune/Co-Management etc The fact is not only are these details of keen interest to other teams and stakeholders, but they are even more of interest to administrator as these fine details can also point out where bigger issues lie as well as being able to constitute for a full comprehensive analysis on a per asset basis.

New Features

This dashboard has a lot more interesting features than the previous dashboards which have been released in the past and below are the overall USP of this new creation;

Live Service Dashboard - This dashboard specifically on the Power BI service will display a detailed summary of everything within you MEMCM estate with each visual being a great representation on giving a SPOG (Single Pane of Glass) illustration on all of your assets.
Full illustrated Resource Explorer - Displaying of all hardware inventory information thus switching logos to represent the assets Manufacturer and Operating System.
Full Cache Compliance - You can now see all of the history within a devices CCM Cache which downloads all of the content specifically for that target device which now displays all folders, files and a full analysis of date trends and sizes.
Real-Time Client Log File Analysis - You can now be able to look at the last 5 - 15 lines of key log files which are created on your assets which are monitored by MEMCM to obtain not only a deep analysis of your assets but also not needing to be present at the machine physically anymore.
Analysis of all Asset Connections - You will be able to see the status of every kind of connection which the asset has access to. For example you can see if the machine can be pinged, can be RDP'ed and Remote Controlled. Others include connectivity status to a CMG, Co-Management, Azure AD Tenancy and client health results status.
Advancements on Network Assignments - Now you will be able to see not only which Management Point an asset belongs to, but you will now be able to see exactly which Boundary it belongs to and which DPs & DP Groups they are assigned to, which allows you to see exactly where its distributions will arrive from.
Enhanced Tooltips - Specific visuals will reveal important tooltips that display more additional data to compliment the specific information in which you are looking at.

How to get the Dashboard Solution

The solution will require implementation from Walsham Solutions so that this can be tailored specifically to each individual customer as many will have different requirements as well as setups when it comes down to your Power BI licensing model.

Pricing is varied as we have two versions in which we can provide the implementation for;

Live Service Version - This would be the dashboard setup within the Power BI service where in addition you would also be able to utilize the Live Service Dashboard below and be able to use your dashboard solution within the cloud. This is a great way of also connecting to your resources if you are using Azure SQL or any other resources within the cloud. Course you can also have gateways setup to your on-premise servers if required so
On Premise Version - This would be more the offline version or similar to which have been released from WalshamSolutions in the past, where the dashboard can be used in a standard Power BI Desktop application perspective and would be using a manual refresh to keep everything up to date which is still also extremely effective depending on your requirements.

Please get in touch on dwalsham@walshamsolutions.com for a further consultation and implementation plan

MEMCM Device Dashboard

Below we will go through each individual report of the overall solution and dashboard and provide great detail as to the overall features and unique selling points of each

Live Service Dashboard

Available for those who wish to utilize the Power BI service report model of this solution, this is the live dashboard which consolidates your overall estate of device compliance which range from various visuals which provide by a summary for each individual report within the MEMCM Device Dashboard solution.

A very special and helpful feature in this case is the Live Bulletin widget which aims to display all of the latest information with your devices which have happened specifically today!

The whole idea was to make it like a news bulletin or today's newspaper to see the latest news, in this case the top headlines will be whats currently happening in your estate right now

This can range from everything such as;

New Devices Added Today - See all of the devices which have just been built today
Devices which have turned healthy - See all of the devices which have turned into a healthy state or are in healthy state
Devices Which have turned unhealthy - See all of the devices which have now an unhealthy state
Devices successfully got deployment - See the compliance of successful deployments to machines today
Devices failed deployment - See the compliance of failed deployments to machines today
Devices detected a virus/cleaned - See the machines which have had a virus detected today

This would greatly help the administrator keep track of everything moving and know exactly whats going on, especially with alarms setup and ready for anything which changes so the administrator is notified for where anything urgent may occur.

The black tiles in which you see next to the company logo are KPIs which display the current amount of devices within your estate which are deemed as healthy, contactable or successful compared to the devices within your estate as well as a minimum target in which ideally we would like to hit which has been currently capped to 80% by default but this of course can be changed

Further than we can see the overall section which provides the following widgets and features;

Chassis Type in Estate - This section will show exactly not only the count of the devices but more specific in terms of the chasis type which can range from Desktops, Laptops, Tablets, Mobile Devices, servers etc. We get alot of cases in which we want to know a specific broken down count on the types of devices we have within the estate so this provides a great summary
Click widget to go to the Device Hardware Explorer report
Manufacturers in Estate - This widget will show all of the different flavours of machines which are currently within the estate to give an idea on perhaps different drivers that can effect each different manufacturer.
Click widget to go to the Device Hardware Explorer report
Operating Systems in Estate - This widget shows all of the OS'es in your estate but it will also outline the build number as well so you can see the fuller complete drilldown on overall compliance of the different types of operating systems in your estate
Click widget to go to the Device Hardware Explorer report
Top Ten Biggest CCMCaches (MB) - This widget will detail which of the top 10 machines within the estate have the largest CCMCache folders which will provide a great help in not only knowing how much space which the CCMCache can be impacted at but also see a download history of the machines in question once clicking the widget.
Click widget to go to the Device Download History
Current Deployments - Simple tile to display how many current deployments are running within the estate in total.
Click widget to go to the Device Collection Assignment
Total Collections - Simple tile to display how many collections there are within the estate in total.
Click widget to go to the Device Collection Assignment
Latest Client Logs Entry - Simple widget which shows the date of the latest client log entry to satisfy the administrator that client logs are indeed being synchronised.
Click widget to go to the Device Client Logs
Boundary Assignments - This widget will show a great drill-down of how many machines are part of each boundary which tells us exactly how and where they will be receiving their policies and deployments from.
Click widget to go to the Device Connections
Client Settings Assignment - This widget shows all of the custom client setting assignments which the total number of devices are applicable to.
Click widget to go to the Device Client Settings Assignment
Windows Defender Signature Compliance - Pie chart widget which shows the signature compliance of how many have received updates for their anti malware today, within 3 days, 7 days and also older than 7 days.
Click widget to go to the Device Windows Defender Compliance
Top 10 Installed Applications - This widget shows a chart displaying the count of the top 10 installed applications throughout the estate and its device count.
Click widget to go to the Device Software Explorer
Malware Detected - This widget shows all of the different types of threats and viruses which have been detected within the estate.
Click widget to go to the Device Windows Defender Compliance

Device Hardware Explorer

First up is the Device Hardware page which is a more refined visual of the resource explorer view which you would get within the Configuration Manager console when looking at specific devices.

On left hand side you can select specific devices and view all of the vital information in which will most likely be asked or requested by many which can now be displayed quickly and efficiently;

Client Version
Last Logged On User
Last Logon Time
Last Hardware Scan
Last Online
Last Policy Request
OU Name

Where the Last Logged on User is concerned, we may choose to drilldown in further into the details so that we have a better idea of usage on the specific device we are interrogating, so to aid with that a tooltip has been added to provide additional information which can be seen just by simply hovering over the Last Logged On User field and the Last Logged on Timestamp field.

The other side of the report then shows all of the vital hardware information, which is grouped in sections which provide categorical detail such as;

Device Type
SM BIOS Information
Operating System Information
Memory Information
Manufacturer Details
Network Information
Disk Information
Processor Information

Neat features about this are things such as the Device Type, Manufacturer & Operating System Information tables where when you select a specific device then the logo will change depending on the type of device with below being an example of a Windows 10 Device

Also another feature added is a tooltip for the Disk Information table, which will pop up with a visual which illustrates a breakdown on all of the data downloaded from SCCM to the CCMCache folder, so you can get an idea as to where you stand with space being occupied

Device CCM Cache History

The download history page focuses on a intricate breakdown of everything downloading to the CCMCache folder. Here we can see the size of the CCM Cache currently and the top 5 biggest folders which are utilizing the most space.

The Cache Folder size matrix gives a great breakdown on each individual file which has been downloaded to that folder within your CCMCache folder as seen below.

Also we have a real-time trend analysis which shows not only the total but a progressive total where we can see exactly when something was downloaded and how big it grew until it eventually gets the total size shown in the charts.

And below is a chart which shows a count of how many files are in each folder to give a better summary.

Device Collection Assignment

The Device Collection Assignment report allows you to investigate all of the collections in which an asset is a member of, a feature in which you are able to do in the MEMCM console now when selecting a device within the Devices view.

However, this page works to consolidate information a little step further with the following below;

Excluded Membership - This report will show you all of the collections in which a device has been specifically excluded from, where certain collections have been excluded on some this will detail exactly which ones it cannot be a member of. Which will save a lot of time in trying to find as to why a device cannot be added into a specific collection
All Assignments and Compliances - Here you can see every single kind of assignment which the device is part of whether it be an Application, Package, Task Sequence, Configuration Baseline or Software Update Group deployment. Here you will not only see a very detailed list but you will see its current compliance state for every single one its a part of.

In addition you can also see a summarized breakdown on the left which gives an overall count of the type of assignments as well as how many collections i am and not a part of.

Device Client Logs History

The Device Client Logs History report is a very different feature which amplifies the uniqueness of this dashboard. So this will detail all of the client logs latest entries which revolve around the latest 5 - 15 lines depending on the log in question which will be synchronized into a consolidated view in which you can select the following;

Device Name
Date Range
Log File

This will allow the user to the latest entries which is perfect for knowing exactly which distribution points that your latest download was pulling from which compliments the Device Download History page where you can refer to and see exactly the dates as to when you received the package and where exactly from.

You will also be able to see the date of the latest log entry and how many entries are there in total for that device, and how many client log files are present to be viewed.

Device Connections

This report goes into great detail over every single type of connection in which a device has into SCCM or any type of remote management in general. Alot of us depend on the client recent activity timestamps, whilst they provide great information may not specifically state exactly whats wrong (or right) with the device.

So this report gives a great overview on uptime of each component.

For example we have the Connection Status table which shows the following;

Ping Connection - Can the machine be pinged currently
RDP Conenction - Can the machine receive a successful RDP connection
Remote Control Viewer Connection - Can the machine be controlled using Remote Control Viewer

This information is imperative for a system administrator who may need to connect to the machine.

Not only that but this has been expanded to cover all of its cloud connection points to see exactly where this device is being managed by.

So tables such as;

Azure Active Directory Tenancy
Co-Management Status
Cloud Management Gateway Status
Microsoft Defender ATP Status

All provide great detail on its uptime and current connectivity to see if a machine is being managed on premise or over the cloud.

The last section is then the Device Site Assignment table. This is very important as a lot of us want to know how does the device connect through into SCCM by which boundary. And boundary issues are extremely common to a lot of issues receiving any kind of deployment from being in the wrong boundary or not in one at all.

So here we can see which boundary its assigned to, including the correct distribution points and management point. And from here we can gauge a completely deeper understanding on the devices whereabouts in the network. The great part about this section is that it applies for all of the boundary types (except IPV6 sadly :) )

Device Client Settings Assignment

The Device Client Settings Assignment report shows all of the custom client settings in which a device is a part of. This time this has been leveraged to SPOG (Single Pane of Glass) view so that now you can see every single setting which is applicable in that custom client setting.

And also to take it a step further, we have also included the ability to look at the default client settings which are applicable to every single device in SCCM so you can see all of those settings and also draw a comparison to both.

Device Software Explorer

The Device Installed History is a further extension to the overall resource explorer, in this case it focuses more on the software side by providing a deeper look into the following;

Installed Applications
Installed Updates
Recently Used Applications
Services Status

Each can be refined by looking at the installations by Publisher and Year to get a greater understanding of what is installed on your device.

Most frequently used applications is also a great way to gauge exactly what a device is currently being used for, and also to compliment is the Services Status which shows all of the status of each service for which ones are started or stopped abruptly.

Device Windows Defender Compliance

The Device Windows Defender Compliance is a deep analysis of all areas when it comes to anti malware.

We recognise that many organisations may still be utilising Windows 8 or Windows 8.1 which would inform us that they are still using Endpoint Protection as opposed to Windows 10 being mainly Windows Defender, so all aspects are indeed covered here.

Firstly we look at the compliance around scans, but also specifically as to whether this device has obtained the latest updates for its Anti Virus signature, where there are a lot of target driven results to hit to achieve full health compliance across your estate.

But not only that another part is the health of the device when it comes to Malware and virus detection. Here we have widgets which detail what the device has and if all is healthy.

And to top it off we also have the custom settings for any EP policy in which the device is applicable to.

Antimalware Settings
This page contains the default antimalware policy which applies to every single machine by default, course depending on the priorities which you have setup this can be a different case. Which is why this page contains a table which shows all of the custom antimwalre settings which this device is also applicable to

Windows Defender Signatures
A matrix table configured with conditional formatting which displays exactly which signature category compliance it currently falls under. The flag will simply change to a different section if its status has changed at anytime.

Power BI - System Centre Dashboard Best Practices

Fri, 24 Jul 2020 10:54:25 GMT

Best way to transform your System Centre Data Efficiently

Purpose of this Article

Over the years you have seen Walsham Solutions create various dashboards and reports from using Power BI, transitioning from the traditional .RDL Report Builder (Paginated Reports) which have provided even more enhanced and advanced detail to our statistics and provide great ways of performing health checks.

We normally see Power BI & BI Intelligence evolve around financial statistics and crunching big data in this way, but wanted to bring the focus on the optimisation of a similar standpoint to our System Centre toolsets such as the ones mentioned below;

MEMCM (Microsoft Endpoint Manager Configuration Manager)
SCOM (System Centre Operations Manager)
SCSM (System Centre Service Manager)
SCVMM (System Centre Virtual Machine Manager)
SCDPM (System Centre Data Protection Manager)
SCORCH (System Centre Orchestrator)

These also provide very important statistics for us, especially when it comes to vast areas such as deployments, monitoring, virtual estate and backup information, the same can be done for our tool sets.

However the fundamental differences is that these tool sets normally have a Data Warehouse with very specific data retention settings which can store data for how long we choose to configure, and miss-configured or non configured data retention on a Data Warehouse as well as an operational database can cause data source access and data transformations to be incredibly slow. This has a knock on effect for whenever we try to refresh data in anyway possible, which then has a negative impact on our database infrastructure as well as resources.

So this article is to illustrate a guide of best practices and what I would recommend to use and the reasons to backup each recommendation

Spring Clean your System Centre Tools

This is the most important prerequisite for preparation before developing any reports or dashboards.

Old or orphaned data clogging up your databases have a huge effect on how you can transform your data. Below is a section for each System Centre tool to give an idea on what to do to keep old data out of scope.

MEMCM/SCCM

Your site maintenance tasks can clear a lot of your old data within your environment. Some are enabled by default whilst others may require a little bit more tuning. A full comprehensive list can be found here https://docs.microsoft.com/en-us/mem/configmgr/core/servers/manage/maintenance-tasks

Another part of the cleanup which you can run would consist of the age of old deployments which you have in your environment consisting of software update patching schedules and package deployments.

Also to note that collections with huge member counts can also play a huge factor especially if wanting to build something that has deep analysis into every device in each collection for example sake. There is also a best practice that exists which talks of not using "All Systems" collections as its main limiting collection to avoid adding any machine, but there is also another reason for why you shouldn't. Any time in which you refresh the All Systems collection or does this by default around 4:00 am, every collection that uses this as its limiting collection will also update and if you have member counts of over 10,000 or more this can lock tables for a while thus hindering your refreshes of reports.
A Data Warehouse point role could be great in the future if wanting to transfer historical data across if you still require it, but there are also alternative methods.

SCOM

Data retention for SCOM is extremely important for both the operational database and also its Data Warehouse.

Tools such as the DWDATARP are great to set the data retention settings for your Data Warehouse so certain datasets don't get too big and can be cleared regularly https://techcommunity.microsoft.com/t5/system-center-blog/data-warehouse-data-retention-policy-dwdatarp-exe/ba-p/340415
Database grooming settings on the administration tab in your SCOM console should also be set and configured appropriately to ensure that you don't keep more data than you need.
Any management packs which are causing too much noise (configuration churn) should either be removed or properly tuned beforehand

SCSM

SCSM uses a multitude of databases which make up the entire Data Warehouse which consist of several areas of staging data which is organised from the data warehouse jobs.

Ensure any management packs specifically imported from a SCOM perspective are only ones of which you need as the connectors for alerting can have a tendency to migrate a possible configuration churn into your service requests model. In addition where inventory is concerned for your SCOM CI Data connector ensure you are on top of any orphaned objects which are brought across from this connector.
Similar to the SCOM CI data connector, others such as Active Directory and other System Centre tools such as SCCM also have their own cleanup methods which should ideally be adhered to ensure a clean topology all round.

SCORCH

Orphaned runbooks and logs can cause huge impacts on the Orchestrator Database. Clearing of the authorization cache on a generic workaround wont always work you will need to ensure your log clearance settings are also configured correctly.
Ensuring you don't have an overloaded amount of runbooks creating too many runbook jobs which may not be needed.
Any enhanced logging which consists of logging every single step of a runbook could cause tables to be extremely big, so handle this with care if looking to bring this into a data model on your report/dashboard.

Pre Data Transformation

Selecting the correct Data Source Connection Type

DirectQuery Not Recommended

Apologies if this section re-iterates what you may already know about certain connection types. This is only to give a breakdown as to the reasoning of the recommended type to select.

But for those who may not be aware there are a couple of data connectivity modes to be aware of when creating a data source to your SQL Databases when creating any reports or dashboards.

For those who have created a report using SQL report Builder (Power BI report builder now for paginated reports) will remember its layout as where it will show all of the visuals and tables which you used to create the report, and nothing would show until you actually ran the report at that specific moment where the data is ran in memory? In Power BI this is similar to the DirectQuery data connectivity mode.

Utilising this would give you a similar option, however the difference is that the latest data would be shown all of the time which is a great feature, but there are various limitations which may prevent you from using this particular connectivity mode below;

A one million row limit - We know as System Centre admins even with the proper maintenance this is not realistic in a real life scenario to maintain with. If we take tools such as SCCM where one would have various deployments over a span of several months, tables such as VSMS_ClassicAssetDeploymentDetails alone would rip right through this and the downside would be that key information would be missed.
Only one Data Source can be used - Another hurdle here. If we take for example tools such as SCOM and SCSM, where they use a Data Warehouse or in fact multiple data warehouses in SCSM's case. We would be unable to manipulate data which would mix between multiple DB's to create a complete picture for our Dashboards
Time & Date Capabilities not available - This prevents us from looking at our data on a historical perspective, which is very key when it comes to root cause analysis or stats which we would need to find out spanning over certain periods of time.
Single Direction Relationships - Not that I can categorize these points as to which is the most impactful to providing our targeted results, but this potentially could be the biggest killer. We will go into this one in more depth but this feature stops you from having certain datasets having a relationship which allows you to filter between results, look at it similar to a SQL JOIN function.

I would say if you have a report in mind in which you would require the latest data and is not dependent on the points above then DirectQuery is OK, but it wouldn't work well if you are trying to make a more complicated report or dashboard. Its benefit is that its beneficial for large datasets, but in our sector large datasets need to be transformed more.

Import is the Best Option

The standard importing of data into the report/dashboard is more beneficial for us as it's a lot quicker than the DirectQuery mode but also there is another point which would benefit System Centre Admins.

When it comes to these tools there are a lot of refresh and synchronization points which have to take place before the data in the database is refreshed and then shown within the respective GUI consoles after. Where the import data connectivity mode is helpful is that depending on the last time we refreshed, we can see all the data refreshed from the last time we performed it so we can still observe the last synced point of our SPOG (single pane of glass) data.

Also without the limitations of the above this provides a better standpoint for an administrator.

Do Not Over complicate Queries

Over complicated SQL queries can really play an ever bigger part in affecting the performance of not only your dashboard but easily eat up all of the resources within your SQL Servers.

I have learned that certain lines or conditions in which you add to your queries can cause your SQL TempDB to rapidly increase and fill the designated drive up until you receive errors of timeouts and out of space errors.

Below are some of the conditions where if you can avoid them at all costs especially if they are tables, views, functions or stored procedures which have a great amount of data in them;

Distinct
Where (Basic conditions are fine multiples which require bracketed lines will cause issues)
Order By
Group By

Bear in mind these conditions are very common in SQL language. And these don't really cause issues if your tables are not that big. But if you have a query which can have tens or hundreds of millions of rows, then these conditions have to go through every single row before its able to respond back with results which can take a long time, and during that time not only can timeouts occur but the TempDB will go into overdrive loading the data into memory and will potentially wipe out the disk that looks after it.

With the transformation of your data where possible is to let all the data unfiltered come through from the data source so that we can utilize the Power Query and M language to filter the same way as we would have done when compiling a more detailed query, which is the true power that we have using Power BI.

How to overcome huge historical tables

In situations where you use a tool such as SCCM where you have historical deployments across a multitude of collections with huge member counts, there maybe times where this is needed and simply removing them or cleaning them up is simply not possible.

There is a workaround for this. These tables and these queries can consist of hundreds of millions or potentially billions of rows, and there may not be a justification to drill down into data that would be too old so we ideally want to keep the information in the report/dashboard somewhat current.

The best workaround in this situation is to add a WHERE condition which adds a date range which should substantially reduce the amount of rows that you will be bringing into your dataset.

Using SCCM for example if i have package deployments which go back as far as 2017, what i may suggest is to add a line such as

WHERE vsms_ClassicDeploymentStatus.DeploymentTime >= dateadd(day,datediff(day,0,getdate())-365,0)

The key line is the DateAdd condition which allows us to cap the date range of data we want to bring into the data source. In a more ideal world if the tables or queries didn't contain too many rows we would have used the Power Query language to add this range, but if we are having issues in which timeouts occur this is potentially the best way to have your interpretation of only current data in your data set.

For any data which you wish to view older than this could potentially be used with the default paginated reports in your respective toolset.

Here is a reference for more on the DateAdd condition https://www.w3schools.com/sql/func_sqlserver_dateadd.asp

Turn off Background Data Loading

This is widely covered but has to be re-emphasized that this should be turned off.

The Background data option is when you have data previews which are loaded in your queries, these happen in the background by default whenever you import new data. If you have datasets which are very big not only can the preview refresh halt the amount of time it takes for your refreshes to change status from evaluating to loading data, but your TempDB will also go through the roof once again.

To change this perform the following on your report/dashboard using Power BI Desktop;

1. Go to File - Options and Settings - Options
2. Go to Current File - Data Load.
3. Click the tick box under Background Data - Allow data preview to download in the background.
4. Click OK

Post Data Transformation

Below are sections of what to do with the data which you have now imported into your report/dashboard within Power BI. Depending on the size and the age of your data will determine which of the following sections would be applicable to you.

Relationships, Appending & Merging Scenarios

When it comes to filtering and consolidating your data together then these types of sections are very important especially for System Centre Data.

I will provide some examples from of the dashboard which I have created which utilise these methonds.

Relationships
So lets start with relationships. For those who are familiar with SQL conditions such as "JOIN", "INNER JOIN", "LEFT OUTER JOIN" then you will be quite familiar to how the structure works for Relationships. Matter of fact it applies quite well to Append Query and Merge Query as well but we will go through the differences.

Now with Relationships these determine how data is cross-filtered between two datasets and are joined together by a column from each dataset which has a common value.

Below is a screenshot of a relationship which I have.

So where the cardinality is concerned i tend to use Many to One which is really mapping values in one dataset that have the same value multiple times to another dataset which only contains unique values.

For the cross filter direction i normally use both as where using visuals such as dropdown lists and combined visuals it allows me to cross filter between both.

One of the recommendations i would point out is to make sure you create a dataset specifically for any reference objects such as;

Devices
Groups
Collections

Anything applicable in a sense where CI's will be referenced as a centralised point. Reason being is because you may run into issues where you get errors on the relationship window such as "You cant create a relationship between these two columns because one of the columns must have unique values" error. And these are so common when we have tables that can display the same object multiple times but only one small column is the reason why that row is unique.

So for example in this SCOM screenshot above, i made a dataset called "SCOM Groups" which only contains the SCOM groups as unique values. In this case you do have permission to use "distinct" if needed as these tables aren't very big .

Append Queries
Appending queries is the equivalent to a concatenate condition, by merging the tables together for which have the same column names as each other.

This function is quite handy especially when wanting to breakup datasets which can be substantially large, and to avoid additional resource impact on the database which you are extracting from and to just focus on the data transformation with Power BI is more beneficial.

Here is an example of appended query.

In this example this is to append a dataset which contains each device in each collection and the software update deplolyments applicable to each device and its collection membership. As these columns are the same they can be appended/concatenated together to show an overall bigger picture of the compliance. Forming this as one dataset caused almost hundreds of millions in rows, so in this example an append query was best and provides the output below.

Please handle this with care, as this can seriously increase the amount of rows your refresh will have to deal with when it comes to your dataset, so i would suggest you only use this method only when you don't have a great deal of historical data to run through.

Merge Queries
Now merge queries is basically merging everything together. This is the equivalent to a "LEFT OUTER JOIN" condition.

Same concerns apply where it can seriously increase the amount of rows processed. However in this case you are able to utilise better conditioning for columns you want to create the relationships between. So you can look at it as a more advanced version of a relationship.

For this particular example this is for SCOM, where it merges a dataset which contains all of the monitoring class instances of each managed device within SCOM, to all of the applicable SCOM monitors which apply to every single monitor and specified management pack. As its a merged function all columns will be joined up and show as "blank" where not appropriate as its using a "Left Outer" join kind.

Formulating a dataset for all monitoring class instances and each device its applicable to and each monitor and management pack would cause a very huge dataset, so this is a great way of overcoming this.

Grouping

Grouping is also another way of filtering your data down even more, and again is quite similar to the Report Builder when creating a table within the table wizard.

Here is an example of grouping on my SCOM Dashboard

For this particular example of grouping, this is a dataset which contains every single alert and historical alert within SCOM. Where the grouping plays a part is selecting the key columns which we require.

Now with the grouping in this case we wanted to obtain the alerts which have the latest raised date and also the latest resolved date and the latest resolved date, this way we can see the latest occurance of the alert and see every single resolution state it entered until it eventually reached into a resolved state as seen below.

Standard Table vs Elaborate Visualizations

One of the elements which shine to myself and those who have used paginated reports, is the Table Wizard formats in which you can create a report where there is specifically a table of results when running the report to execute your dataset query.

For System Centre Admins these can be quite important for those which provide reports on a regular interval whether weekly, bi-weekly or monthly.

Here is a screenshot of a table view within a paginated report.

Here is a screenshot of a table within Power BI

Here is a screenshot of a visualisation type report detailing results you can see from a table

The point really being there maybe scenarios in which it maybe more convenient for you to have a page which is just a table, as opposed to a page which has multiple visualisations which you or a user which has been provided access to and have to right click and click something like "See Records" or "Show DataPoint as Table".

The result of this is similar to creating a linked report action in Report Builder.

Refresh Schedules

Power BI Desktop only has the ability to manually refresh when you specifically require one. The best way to utilise refreshes on a schedule is to use Power BI Report Server, or using the Power BI service (though may require you to install a gateway https://powerbi.microsoft.com/en-us/gateway/ )

It's important to know the peak times of when your system centre tools require a refresh so you can capture the information you need at the most appropriate times. Some tools such as SCCM where they have evaluation schedules will determine when the data will refresh so this can play a part in your refresh schedule also.

Other tools such as SCOM, SCORCH etc will be a bit more on demand with their data. SCSM on the other hand has Data Warehouse jobs which execute the data stagings as well as the connectors which import data into SCSM at particular intervals, so having a well thought out plan for your refresh schedules is key.

Also bear in mind if using these in Power BI service that there is a limit of 8 refreshes per day if using a free or pro license. But if you happen to have a Premium license then you can have this up to 48 times per day.

DAX Example - Custom Tables for custom definition

There are some common DAX commands which I use within my dashboards and reports. These are quite similar to the formulas which you may come across in the Report Builder however these can vary so here are some ones which I have used and the benefits in which they provide.

COUNTA
So this is great for when it comes to providing stats for table, filter card and some chart visuals. COUNTA is really to count the cells which dont have blank or empty (null) values.

This example is for SCCM ad determines a measure which calculates how many machines have a "Scan Failed" state.

Custom Column DAX

There maybe times when you want to have some more custom definitions in a way which you can group your system centre data objects into a more friendly way of breaking down key stats. So I have an example to explain further.

In my SCCM Deployment Dashboard there is a table which i use which is on the Collection Severity Forecast page. It shows a grouping of stats of all of the devices which online, offline etc But how it was done is key and this was done by creating a custom column using a DAX formula. Below i will give a breakdown on how this works.

So i have a dataset called CurrentlyOnlineMachines which contains a list of all of the machines in SCCM and the timestamp of when they performed a hardware scan last.

So suppose i create my own definition of how i perceive to be an "Online" machine. For this case i would say any machine that has not had a hardware scan within the span of 2 days i would then consider to be offline.

So then i create a new column with this forumla here

Now we have a custom column created by a DAX formula that states any machine which has not had a scan within 2 days is offline, and anything which is online has.

So now when i want to have a result of how many machines are online i then create a new measure based on a DAX query which uses the value specified in the DAX formula value above.

And now to take this a step further with more additional conditions added to a DAX query, i now want to work out what i would define as a Success criteria.

Now if you look at the current forecast table above we have online machines, offline machines, no hardware scan machines and failed wsus scan machines. Each of them created with a very similar COUNTA dax formula, we want to summarise this as one big measure. In order to do that we create the conditions to build our success criteria as seen below.

You can see here we add conditions where the machine does not show a failed scan state and also excludes machines that have no hardware scan. So this builds our success criteria.

So now we want to define a success rate based on an overall percentage. For this we perform a mathematical measure which divides the SuccessMachines criteria by the total devices that are in the specific collection within SCCM.

And to ensure the format is correct we configure the following settings in the ribbon bar above

Splitting Up Dashboard to Reports

If you have a dashboard that perhaps contains multiple pages, though the convenience is that you have all the information you require in one centralized dashboard, the danger is that some of the datasets in which you have can be susbstantially larger than others, whilst some can be quite small.

Having a mixture of both of these especially when performing a refresh can cause the refresh rate to be slower and slower, especially if you are wanting to know specific information on one of the pages more critical than others.

If this scenario fits you, analyse which pages are most critical to you as well as which may be the biggest in terms of DataSet row sizes and look to which ones you will want to split up.

This way you can have a refresh schedule set on your Power BI Report Server to be more optimized with the focus on datasets which are specific to the report you have made as a singular report.

Summary

Overall these are the guidlines and best practices which i've developed when it comes to optimizing the output of data retreivied through the System Center toolsets to display the most accurate data possible and without throttling resources on the SQL infrastructure.

There will be more additions to this article as we go along and also welcome other points, tips or best practices which would indeed help the optimisiation of System Centre data

System Center Configuration Manager Deployment Dashboard V2.0

Sat, 28 Mar 2020 12:37:38 GMT

Back and Revamped with greater features!

Introduction

Here is the new and improved version of the System Center Configuration Manager Deployment Dashboard with its new release being labelled as V2.0

It's origins come based on the first version of this PowerBI dashboard and this was then based on the original Report Builder version which can also be downloaded on the TechNet gallery - https://gallery.technet.microsoft.com/SCCM-Deployment-Dashboard-63baf9b6

This follows on from the previously released second edition of the Software Update Dashboard which has had a huge make over with additional important features added which will be explained in each section.

Universal New Features

Revamped Design - To make it stand out more and pop especially with vert business critical information there is now more focus on the areas which are needed to be focused on, especially when issues become critical or soon to be critical
Query Reduction - The amount of queries used have now been reduced to speed and optimize the overall dashboard, so any refreshes should be a lot quicker going forward
Query Organizations - If needing to change anything on the dashboard all of the queries have now been sorted into folders which are specific to each page so the organisation is alot better.
Reduced Relationships - Relationships between tables has also been optimized to provide better performance.
Restructured Queries - Have sorted through various queries pulling information which again is alot more enriched with important information and columns not in use have been minimized.
DAX Enrichment - More DAX functions have been used to provide better statistical analysis overall

Where to Download

The dashboard can be downloaded on the website right here

https://irp-cdn.multiscreensite.com/c1f89e50/files/uploaded/SCCM%20Deployment%20Dashboard%20V%202.0.zip

You can also obtain the dashboard from the PowerBI Dashboard Solutions tab from the dashboard gallery

Software Library Compliance Page

Here is how the older version looked of this page

Below is now the new improved version of the page

New Features for the page

Consolidated Visual for Software Library - Now you can see the categorized structure of all of the content which you have in your Software Library which is also reflective of how it's categorized within the Configuration Manager console. Also using the drilldown facility allows you to look into the manufacturers of each if wanting to see a more granular search
Software Library (CI Assets) - Here you can see a count for all of the individual software updates and also the drivers which are within your software library estate just to make it cleaner from the visual on the side.
Top 5 Applications Visuals - Revamped from using a table to see a list of them now you can see an exact count of the most installed applications per user or device and in a more appealing visual way
Software Dependant on TS Visual - Here you can see which applications are part of a Task Sequence. Of course to get a better understanding of which task sequence they are a part of is best to go to the Operation System Compliance page for this
Enriched information for exporting - Each of the categories contain the folder path to the object within the console for those who have complicated folder structures in your consoles.

Collection Severity Forecast

Here is how the older version looked of this page

Below is now the new improved version of the page

New Features on the Page

Current Forecast Table - So similar to the second release of the Software Update Dashboard this contains a forecast made up of all the devices, online/offline and those with no hardware scan and will work out what the success rate will be of this potential deployment. The legend table gives a breakdown as to what all of the attributes mean in greater detail
Departments/OU Severity - This has been revamped now to show only the OUs which the devices are specifically in as opposed to the entire tree making it more complicated to understand the stats, and also showing which users/departments are also affected. The severity manual part has been removed as better visuals give it a better look into the collections.
Additional Slicers - Now can break down into the Collections, OUs and Departments if wanting to be more granular.
Collection Membership Per CI - This is a new feature added! Now when we have several deployments and collections going and we have absolutely no idea what machines are a part of, this has now changed! So you can select between Devices or Users and selecting a device now tells you the following;
Total Collections its a part of
Deployments going to that device/collection
It makes investigating so much easier :)

Application Deployment Compliance Page

Here is how the older version looked of this page

Below is now the new improved version of the page

New Features on the Page

Additional Slicers for Granular Searching - Sort by Assignments and collections to alter all of the information on the page
Online Devices Only Slicer - This is similar to the feature added in the second release of the Software Update Dashboard where you can limit the compliance to only Online Devices.
(Note: if wanting to see all devices just click nothing for All, choosing false will filter it by only offline devices)
Summarization of Deployment Changes - Instead of showing it as a numbered statistic, you can now see the records of each status of deployment for each application displayed within the visual.
Affected OUs & Devices - Better analysis on the OUs in which the devices reside in which is great when it comes to analysing which GPOs could or could not be causing an issue. Departments showing the departments of the primary users which it affects.
Check Device Compliance Status - New feature added in which you can select any individual machine within the deployment and you can see what status the deployment for that machine is in, which is alot easier than searching through the exported records or any other means.

Package Deployment Compliance Page

Here is how the older version looked of this page

Below is now the new improved version of the page

New Features on the Page

Additional Slicers for Granular Searching - Sort by Assignments and collections to alter all of the information on the page

Online Devices Only Slicer - This is similar to the feature added in the second release of the Software Update Dashboard where you can limit the compliance to only Online Devices.
(Note: if wanting to see all devices just click nothing for All, choosing false will filter it by only offline devices)

Summarization of Deployment Changes - Instead of showing it as a numbered statistic, you can now see the records of each status of deployment for each application displayed within the visual.

Affected OUs & Devices - Better analysis on the OUs in which the devices reside in which is great when it comes to analysing which GPOs could or could not be causing an issue. Departments showing the departments of the primary users which it affects.

Check Device Compliance Status - New feature added in which you can select any individual machine within the deployment and you can see what status the deployment for that machine is in, which is alot easier than searching through the exported records or any other means.

Software Update Deployment Compliance Page

Here is how the older version looked of this page

Below is now the new improved version of the page

New features on the Page

Additional Slicers for Granular Searching - Sort by Assignments and collections to alter all of the information on the page

Summarization of Deployment Changes - Instead of showing it as a numbered statistic, you can now see the records of each status of deployment for each application displayed within the visual.

Operating System Deployment Compliance Page

Here is how the older version looked of this page

Below is now the new improved version of the page

New Features on the Page

This page has had a complete revamp!

Now for what it doesn't necessarily doesn't make up as visually appealing (maybe down to the fact i have one Task Sequence with one deployment :) ) it does make up in very huge detail with the below bulletpoints!

Machines Built within X - This shows all of the machines that were built within 24 hours. Course if needing to track a more historical background this can be done too using the filter slicer
Current Machine Build States - This is a fantastic feature and also completely revamped! So for the last version it tracked the entire task sequence, though was great for detail may have looked confusing or a bit too much information. This time it shows the exact step its on, and you can see whether it succeeded in building or if it failed on that step which allows for great tracking whether using log captures or trying to watch the machine manually.
Task Sequence References - You have an additional slicer which controls this now so you can see exactly which package is inside a Task Sequence which is great for troubleshooting if you run into any issues in which the task sequence will not run due to a package ID it is complaining about

Upgrade Deployment Compliance

Here is how the older version looked of this page

Below is now the new improved version of the page

New features on the Page

Configuration Baseline Compliance Page

Here is how the older version looked of this page

Below is now the new improved version of the page

New features on the Page

Additional Slicers for Granular Searching - Sort by Assignments and collections to alter all of the information on the page

Summarization of Deployment Changes - Instead of showing it as a numbered statistic, you can now see the records of each status of deployment for each application displayed within the visual.

Troubleshooting

Timeout Issues

If experiencing any issues in regards to queries timing out then ensure you edit the query and enter a time frame which is suitable for your command timeout properties to ensure it runs correctly

Open External Data - Edit Queries
Go to the query on the right hand pane select the wheel icon where it says source
Enter the Command timeout properties then click OK
Then in the top right hand corner select Apply and Close

Online/Offline Devices Setting

This is currently controlled by a DAX query column called "HWOlderThanXDays", if you wish to change the value to something different to decide how you determine online/offline devices to the following;

(Note: Most of the queries with "Assets" on it and the "System OU Collections" datasets have this configuration)

Go to either dataset listed above on the right hand side
Dropdown the columns and double click the "HWOlderThanXDays" column
You should then see the formula dropdown, change the value from from "2" to another value (you can also change the name of the column too which is before the "=" sign
Click the tick sign to approve this

Queries Complaining of "Departments" Attribute

As the dashboard has a utilisation of obtaining affected OUs and more specifically departments, an important attribute required in this is that the Users discovery method is inventorying the "Departments" attribute.

If you get any issues regarding this perform the following.

Open the Configuration Manager Console
Go to Administration Tab
Go to Hierarchy - Discovery Methods - Active Directory Users Discovery and right click and go to properties
Go to the Active Directory Attributes tab and add the "Department" attribute on the left by clicking Add
Click OK to accept the changes and select the "Run Full Discovery" button or wait until the next schedule runs.

Once this has been populated if you run a Refresh of the PowerBI Dashboard this should now work

Queries Complaining of "Departments" Attribute (Continued)

If you experience issues in which the "Departments" attribute has been enabled but you are still getting errors run a query in your SQL Management Studios "select * from v_r_user"

If your column is showing as "Department" and not "Department0" then there is a fix for this

Open PowerBI Desktop Application
Open the Dashboard
Click the External Data - Edit Queries
Go to the query which has an exclamation mark and click it. In the right pane go to the Source (top option) and click the wheel to see its query.
Look for a line which says " v_r_user.department0 " and replace it with " v_r_user.department As Department0 "

As the dashboard depends on the department0 naming convention this should fix the problem. The reasoning for the difference of the names when it comes to enabling attributes to be discovered from your discovery methods could be down to perhaps the collation of the SQL Instance. But in case you do have this issue follow the steps above to resolve.

System Center Configuration Manager Software Update Dashboard - Version 2.0

Fri, 20 Mar 2020 10:04:43 GMT

Huge advancements made!

Here we have the brand new and improved version of the System Center Configuration Manager Software Update Dashboard. This is very similar to the first one which came out which is still available here https://gallery.technet.microsoft.com/SCCM-Software-Update-8db3e98d

And of course the original version was based primarily on the report builder version, which can also still be accessed here https://gallery.technet.microsoft.com/SCCM-Software-Update-2dbd8d6b

This dashboard has had a completely different look altogether, though displaying the same information however a lot more enriched with detail, information and great visuals which will allow you to pull great exported reports from.

I will step through each change so that we can see the comparable differences and i will also list the new features according to each individual page

Universal New Features

Revamped Design - To make it stand out more and pop especially with vert business critical information there is now more focus on the areas which are needed to be focused on, especially when issues become critical or soon to be critical
Query Reduction - The amount of queries used have now been reduced to speed and optimize the overall dashboard, so any refreshes should be a lot quicker going forward
Query Organizations - If needing to change anything on the dashboard all of the queries have now been sorted into folders which are specific to each page so the organisation is alot better.
Reduced Relationships - Relationships between tables has also been optimized to provide better performance.
Restructured Queries - Have sorted through various queries pulling information which again is alot more enriched with important information and columns not in use have been minimized.
DAX Enrichment - More DAX functions have been used to provide better statistical analysis overall

Where to download

The dashboard can be downloaded on the website right here
https://irp-cdn.multiscreensite.com/c1f89e50/files/uploaded/SCCM%20SUP%20Dashboard.zip

You can also obtain the dashboard from the PowerBI Dashboard Solutions tab from the dashboard gallery

WSUS Sync Catalog Compliance Page

Below is the old version of the page in the previous dashboard

And here is the new and improved version of the page

New Features for the pageNew Features for the page

Revamped breakdown of the synchronized updates - Ability to see not only the update categories, but now will provide a drill-down to the actual products of each category
Synchronization Time Frame Slicer - Now you can structure the time period in which updates were synchronized from the newest to the oldest with the totals being displayed for updates synchronized in total, and a categorized total of the latest updates and all updates which were declined
New slicers for better filtering - Slicers added to see specific update categories, specific products which were synchronized and also to locate specific windows update to check if it is declined or not.
Improved WSUS Error Handling - Status messages provided for when any issues arise with the latest synchronizations.
Improved see records function - With more enriched information you will see better information when performing any exports of data.

WSUS Cleanup Health State Page

Below is the older version of this page

Here is the new improved version of the page

New Features for the page

Obsolete Objects Donut Chart - Same information displayed for Obsolete Updates, Dead Deployments, Failed downloads and computer objects this time more simpler in one chart detailing the total and percentage of the overall fragmentation of the WSUS environment
WSUS Services Conditional Formatting Alerts - Here we have the table of all the WSUS services however this time we have conditional formatting enabled to indicate when services are running and not running, which should draw attention to a system administrator if the WSUS service is down.
WSUS Event History Table - Here newly added is a table which shows the event history so now you can see all status messages with anything being done on WSUS such as when synchronizations start, complete and fail. Great table especially when the Software Update Point (SUP) has triggered a synchronization whether scheduled, manual or via ADR (Automatic Deployment Rule)
WSUS Legend Tables - Legend tables have been added to explain every attribute in finer detail as well as providing recommendations and links to any more information which can help diagnose or resolve an issue overall.

Software Update Deployment Forecast Page

Below is the older version of the page

Here is now the new improved version of the page

New Features for the page

New Improved Forecast Table - The forecast banner is quite an integral piece to this page to provide a summary of what your deployment success rate will be. This has been changed to now represent the overall percentage compared to the total devices of the collection which has been specified. DAX formulas have been used to calculate the success rate being online devices that haven't failed WSUS scans etc
Forecast Legend Table - The legend table provides a breakdown of each attribute within the forecast table which discusses the definition of each and how they are calculated.
More Forecast chart visuals - Here is provided more charts to give a wider range of categorical scopes to further analyse the data to see where different device statuses are located which can effectively work positively or negatively to your success rate percentage
Additional Slicers - Slicers have been added to configure the charts alot better if wanting to zone into more specific information.
Client Version Informatio n - Every single SCCM client version from the newest to the oldest has been accounted for so if you have a wide mixed range of SCCM clients on different versions we will be able to see this.
Windows 10 build information - Similar to the client version all Windows 10 build versions from the latest to the oldest have been accounted for.

Windows Defender Definition Updates Compliance

Below is the old version of this page

Here is the new version of the page

New features for the page

Additional slicers for better filtering - Slicers have been added to further customize the experience for the system administrator to drilldown to more specific information regarding definition update compliance
Display of latest definition updates being deployed - This banner has been advanced so now it will show the latest definition which is currently being deployed to your machines. Note it will only show the latest definition update being deployed as per your ADR rule which is pushing it out
Recalculated way of checking latest signatures - With their both being a statistics table of signature updates as well as per asset statistic table the pie chart now reflects the table below to give a more accurate drill down as to what machines have the latest definition updates.
Online Devices only slicer feature - Added a nifty little feature here. Now when we have SLAs to keep to we know that machines that are not contactable for periods of time or offline completely can affect this, so therefore ive created some DAX queries which can filter out any offline machines to narrow done on the online machines.

Current Software Update Deployments Page

Below is the old version of the page

Here is the new version of the page

New features for the page

Update Status Compliance Visual - Here you can see the status of each update within the assignment and see the compliance status whether they have updates installed, failed, in-progress or even pending a reboot.
Update Compatibility Visual - Here you can see the compatibility of all the updates applicable to the assignments currently in deployment such as whether they are Installed, Required, Already Present and if they are missing.
Machines with Missing Updates Count - Each machine which is involved within the collection or assignment deployment will display all of the updates which are currently missing from the machine
Software Updates Deploying Table - Revamped to display all of the updates involved within that assignment and a statistical breakdown of all of the machines and its compatibility status count
Additional Slicers - Slicers added to customize the collection view and also if wanting to search for a specific update.
Online Devices only slicer feature - Added a nifty little feature here. Now when we have SLAs to keep to we know that machines that are not contactable for periods of time or offline completely can affect this, so therefore ive created some DAX queries which can filter out any offline machines to narrow done on the online machines.

Sofwtare Update and Device Compliance

Below is the old version of the page

Here is the new version of the page

New features for the page

Update Status Compliance Visual - Here you can see the status of each update within the assignment and see the compliance status whether they have updates installed, failed, in-progress or even pending a reboot.

Update Compatibility Visual - Here you can see the compatibility of all the updates applicable to the assignments currently in deployment such as whether they are Installed, Required, Already Present and if they are missing.

Applicable Assignments - Here is the new visual which shows every single assignment applicable to the device chosen

Troubleshooting

Timeout Issues

If experiencing any issues in regards to queries timing out then ensure you edit the query and enter a time frame which is suitable for your command timeout properties to ensure it runs correctly

Open External Data - Edit Queries
Go to the query on the right hand pane select the wheel icon where it says source
Enter the Command timeout properties then click OK
Then in the top right hand corner select Apply and Close

Online/Offline Devices Setting

This is currently controlled by a DAX query column called "HWOlderThan2Days", if you wish to change the value to something different to decide how you determine online/offline devices to the following;

(Note: The following queries "CurrentlyOnlineMachines" and "ClientSummary" datasets have this configuration)

Go to either dataset listed above on the right hand side
Dropdown the columns and double click the "HWOlderThanXDays" column
You should then see the formula dropdown, change the value from from "2" to another value (you can also change the name of the column too which is before the "=" sign
Click the tick sign to approve this

Power BI – 30,000 Row Export Limit Workaround

Mon, 16 Mar 2020 11:27:16 GMT

Learn to export all of your data without restrictions

Introduction

In Power BI, when creating any kind of reports, dashboards which will contain an X amount of rows of data in which your queries will be pulling, a lot of us will want to be able to deliver an exportable report in which we can share.

So once going to a specific visual or in most cases a table visual we click the … ellipsis button in which we are presented with the following options below;

If you have over 30,000+ rows of data in your visual you may see the following notification message below;

Once clicking the OK button you will then be given the option to save the .CSV file to your desired location, and once opening you will see the export has only a maximum of 30,000 rows.

Why do I have this issue

The reason for this is because Power BI Desktop has an ultimate maximum of rows which can be pulled from an export which is 30,000 rows and this is because a .CSV file has this maximum within Power BI.

A .XLSX export on the other hand can have an export of 150,000 rows which is 5X better than the .CSV but the only issues is that you cannot by default do this within Powe rBI Desktop and also if you have data which exceeds 150,000 rows this can still cause issues.

Export Workaround

It’s quite a basic workaround but it does allow you to perform an export of every single row which you have.

On the left hand side you will see the pane which contains three types of views, the first of course being the main visuals you see, but the second option is the Data view which shows you a table of the entire data in which you have which is currently summarized within the Visual view.

Here you can see the entire table data view of everything.

If you right click any column for example and select Copy Table, wait for a few seconds whilst the time loading icon takes its course.

Open up a new excel spreadsheet and click paste.

Here you will now see all of your data and that’s including every row and every maximum row barrier exceeded.

In the bottom right hand corner you can see the count is 47357 which is over the 30k rows maximum

System Center Operations Manager Compliance Dashboard

Fri, 21 Feb 2020 13:05:15 GMT

In-depth look of your monitoring estate in PowerBI real-time!

Introduction

This requirement and solution was formulated over a long period of time in which we may have been tasked to provide a lot of information regarding current monitoring as well as historical monitoring, which can range from various sectors whether more locked down for security purposes or utilising a centralized solution to provide a single pane of glass view into the health of the current estate.

Here we present the System Center Operations Manager dashboard - could have thought of perhaps a more fancier name but as it serves as an all purpose dashboard the name seems to stick.

Why not use the HTML5 Dashboard in SCOM Web Console

There are various widgets in which the same information can be provided in some of the pages in the dashboard, the reason being to separate from the web console was so that we have a source in which for resiliency where if for whatever reason SCOM (primarily the databases) are non-contactable, with the most recent refresh we would still be able to maintain somewhat current as well as historical data.

We may also find a lot of information requested specifically in report format tend to be off the cuff requests, whilst SCOM is a very powerful tool it does require customisation to fit the needs of the requester or organisation. So this dashboard is targeted to provide a standardised answer across many frequent requests

Where to Download

This dashboard can be downloaded from Technet Gallery from the following URL link

https://irp-cdn.multiscreensite.com/c1f89e50/files/uploaded/SCOM%20Compliance%20Dasboard.zip

Dashboard Pages

Front Page

The front page being primarily an introduction page and providing a table of contents, each with providing a brief summary of what each specific report covers.

All text boxes also contain an interactive button which allows you to click and will automatically take you to the desired report page within the dashboard.

Operations Manager Health Compliance

To kick things off we have the Operations Manager Health Compliance which is essentially a SPOG (Single Pane of Glass) look into the overall health of your SCOM Architecture.

We can see the following statistics on the left of a count of managed SCOM Agents;

Healthy Agents
Agents in Warning State
Agents in Critical State
Greyed out State
Agents Not Monitored

We also have charts which show the peformance drilldowns over any specified time period given by the end user which ranges between the performance of Management Servers and Data Warehouse processes.

In addition there able tables on the right which show any alerts which are specific to the SCOM architecture as well as the latest event logs which come from them.

Below is a table which shows the health state of the Data Access Service in case there are any issues with SCOM where you are unable to access it.

Monitoring Compliance

Here we have the Monitoring Compliance page which shows a chart giving a statistic breakdown of all of the SCOM managed agents within your estate. The chart gives a yearly historical breakdown which shows how many agents have been installed.

Another chart shows how many SCOM agents are within a pending state if you are perhaps using a manual agent install or is included within an image build which requires an administrator to approve from the console.

There are however several states in which the agent may be in during this chart such as;

Updated Failed - Tried to upgrade the SCOM agent to a higher version
Manual Approval - Awaiting an administrator to approve the agent from the console
Push Install Failed - Tried to push the SCOM agent to a computer unsuccessfully

Underneath is a small statistic table providing a total for all SCOM agent category totals.

Monitoring Historical Alert Analysis

This next report is the analysis of all of the alert history within your SCOM estate. A report which can be altered over any desired time frame.

So you can change it where appropriate to show alerts over a period of time when the alert was initially created and you can also change where you can see the period of time in which the alert was resolved.

Several charts show breakdowns of other specific properties within your alerting;

Severity Count - Shows the number of alerts which have a specific severity assigned such as Critical, Warning or Informational
Resolution - Shows the number of alerts which have a specific resolution state assigned to the alert from the standard resolution codes through to any custom ones which you may have configured
( Note : If you have configured any custom ones you may have to change its friendly name via the SQL query which i can explain in a section below)
Alerts Ticketed - Optional chart if you are using a ticketing system such as Remedy, ServiceNow or SCSM etc. this chart shows how many alerts have a ticketID assigned to it for reference if needing to revert back.

The table underneath reflects all of the alerts shown over your configured time period or if left as default will show everything. Fields included such as;

Alert ID
Alert Created Date
TicketID
Alert Fixed Date
Resolution State
Alert Description
Alert Name

Data Warehouse Health Compliance

This report is a very in-depth look at the health state of your Data Warehouse.

A lot of overlooked issues tend to formulate around the Data Warehouse which keeps your historical SCOM data, and if not configured correctly or if experiencing any high level of data volumes or configuration churns can easily make your Data Warehouse database sky rocket.

There are handy tools such as the SCOM Sizing Tool for planning & also the DwDatarp.exe tool which is a great tool to analyse and configure your data retention settings to avoid this from happening.

This page provides a chart showing all of the Datatsets within your Data Warehouse and shows exactly how big each dataset is and the percentage of the database it makes up with a legend table showing which dataset it is.

Data Retention Settings
This table shows a list of all of the standard Datasets which are used to collect data to place into the Data Warehouse and stores it for a specific amount of time before clearing. This information is similar to what you would see in the DWDatarp.exe tool, here you will see the current data retention settings which you can see if you need to make adjustments to them. You will also find alerts generated from SCOM will reference DataSets or the rule which controls the dataset that places the data into the Data Warehouse have issues, this will allow you to investigate further.

Largest Data Warehouse Tables
This table shows from highest to lowest in MB how big each table is. A great way to analyse to see if any of these would be problematic when it comes to space.

System Center Operations Manager Database Sizes
Small section which shows the sizes of the database for both OperationsManager and OperationsManagerDW respectively.

System Center Operations Manager Database & Disk Health
Small section which shows the databases and where they are located as well as the free space which is left on the drives which they are placed on. There can be times in which when the databases get full for whatever reason or become unavailable, the services for SCOM (System Center Data Access Service) will shut down and cannot be started creating a windows event log stating the databases are full, or may get a warning or critical alert generated in SCOM before this happens. This section will help give an indicator if this is the case.

Monitoring Configuration Per Asset

This report is similar to the export configuration powershell command and dashboard view which can show this information, but in a Power-BI related view.

From a dropdown list of all of the machines in your estate you will be provided the following details;

Management Packs Available - Shows the amount of management packs which this machine is applicable to. Can be seen by seeing the records. Clicking any of the management packs should filter down the Monitors and Rules which are applicable to the asset or group.
Related Instances - Shows the count of related objects/instances to the machine which are also monitored alongside with it i.e. the Windows Server Operating System Management pack would monitor its disk drives so you may see "C:" as a related object which is being monitored.
Health State Per Instance - Shows the health state of each of the related instances including the machine asset itself across all health states of Critical, Warning, Healthy as well as Not Monitored.
Monitors & Rules Applicable - Shows all of the monitors and rules which are applicable to the chosen machine.

Monitoring Configuration Details
This section shows a table of the related information above respectively, so you can see the actual objects which are applicable to the chose machine such as the Management Packs, Monitors and Rules.

Data Volume Compliance

This report has a similarity to the Data Volume by Management Pack default report in SCOM, but is an alternative version and revamped report. A great way to analyse configuration churn more and having an interactive view on where noise is coming from.

Here you can adjust the date range to see all of the data which has been generated and able to see the following information;

Alerts Volume Generated
Performance Volume Generated
State Volume Generated

Each chart also shows the management packs alongside the percentage and number of the data volume which had been generated. Another great feature is that not only are you able to view these statistics from a SCOM Group, but you can also do it by a single machine name.

Underneath each respective chart contains top ten tables consisting of the objects which have produced the most volume for their categories.

Below is finished with a summary table tallying up all of the statistics.

Asset Health Compliance

A combination of where you can also analyse the health of the Management Servers as well. Overall the Asset health page is a way to get an overall look at the health of a specific asset.

Here you will get the following;

Asset Summary - Here you will see everything from its domain, operating system level and also its current hardware resources and network details such as IP Address/es
Current Alerts - A table showing the current alerts which are applicable for that specific asset
Current Utilisation - Similar to the Performance by Utilisation report, where you can see an interactive chart which can go through the CPU, Memory, Hard Disk and Network Utilisation over a specific time period specified by the end user.

Troubleshooting

Queries Taking too Long

Some datasets may be pulling not just 100s of millions but maybe billions! Depending on how big your SCOM datasets are. If you are finding that your queries are taking too long i have left some lines within the query which are commented which allow you to only pull data which is younger than a certain date i.e. pull data for only up to 6 months to a year.

You can remove the "--" from the comments so that you can limit the data you are pulling.
Also be mindful that if using any additional lines such as "Order By" or pulling to much data can vastly increase your TempDB into hundreds of GBs which may force you to perform a restart of your SQL Service.

SCCM – Software Failing with no explanations or misleading error messages

Wed, 29 Jan 2020 10:13:12 GMT

Issue

When trying to install software with SCCM you might find some very interesting error messages which might not make a lot of sense and also experience applications which fail almost immediately whether they are deployed as available or required deployments.

One error message in which you may come across is one which says “ A Prerequisite for an install could not be satisfied”

The error message for this code shows as 0x80073CFD (-2147009283)

These can be found either in your Monitoring – Deployment and deployment summary

Also looking through your client logs specifically the AppEnforce.log you will also see the same error code referenced for the problematic app.

Reason

As the error message describes it appears like it has an issue with trying to resolve the dependency within an application, although the bigger at hand is that you may find the dependency is actually already installed, evidence of this can be seen within the client log called AppDiscovery.log

You may find this error perhaps more common with APPX packages. Just as an example we can see the details for a deployment type dependency

But just to exhaust all avenues there are also the requirements which are set for a deployment type which the error message could also be in reference to but this is also satisfied as well.

Solution

The way to fix this is to examine the current Windows 10 build version which you are using. If you are running a build from Windows 10 1607 or lower, you may find these errors amongst multiple other applications which may fail due to this.

Windows 10 1709 is more stable from most of these errors but going forward it will be down to ensuring your Windows 10 builds are on the latest version to avoid any errors such as these.

Additional Information - SQL Query for Windows 10 Versions

Though you have a view within the SCCM console to see the amount of devices you have which are Windows 10 and which build versions they are, here is a SQL query which can be used as well as placed into a custom report via Report Builder or using PowerBI which gets all Windows 10 devices and build versions and also contains an additional condition to which you can specify the collection name if you have a specific deployment going to a specific collection

select vSMS_R_System.Name0 AS MachineName,

vsms_r_system.AD_site_Name0 As ADSite,

v_CH_ClientSummary.LastPolicyRequest,

v_CH_ClientSummary.LastHW,

v_CH_ClientSummary.LastSW,

v_CH_ClientSummary.LastActiveTime,

v_CH_ClientSummary.ClientStateDescription,

v_GS_OPERATING_SYSTEM.Caption0 As OperatingSystem,

REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(v_GS_OPERATING_SYSTEM.BuildNumber0,'17134','1803'),'15063','1703'),'16299','1709'),'17763','1809'),'10240','1507'),'18362','1903'),'14393','1607'),'10586','1511') as BuildNumber,

vsms_r_system.Client_Version0 As ClientVersion,

v_GS_COMPUTER_SYSTEM.Manufacturer0 As Manufacturer,

v_GS_COMPUTER_SYSTEM.Model0 As Model,

vSMS_R_System.User_Name0 As UserName,

v_r_user.Full_User_Name0 As FullName,

v_r_user.User_Principal_Name0 As Email,

collections.CollectionName

from vSMS_R_System

LEFT OUTER join CollectionMembers

ON vSMS_R_System.Name0 = CollectionMembers.Name

LEFT OUTER join collections

ON CollectionMembers.SiteID = Collections.SiteID

LEFT OUTER JOIN v_GS_OPERATING_SYSTEM

ON vsms_r_system.ItemKey = v_GS_OPERATING_SYSTEM.ResourceID

LEFT OUTER JOIN v_GS_COMPUTER_SYSTEM

ON vSMS_R_System.ItemKey = v_GS_COMPUTER_SYSTEM.ResourceID

LEFT OUTER JOIN v_CH_ClientSummary

ON vSMS_R_System.ItemKey = v_CH_ClientSummary.ResourceID

LEFT OUTER JOIN v_R_User

ON vSMS_R_System.User_Name0 = v_R_User.User_Name0

Where Client0 = '1' and v_gs_Operating_system.Caption0 LIKE 'Microsoft Windows 10%' AND Collections.CollectionName = '<Enter Collection Name>'

SCCM: Windows 10 Feature Update Rollouts via Application Package

Thu, 28 Nov 2019 12:57:30 GMT

Update your Windows 10 estate with an Application method!

For those who are performing the Windows 10 feature update rollouts using SCCM software updates you may experience a multitude of issues such as;

Feature Update showing as compliant - Normally resolved from having the latest version of the SCCM client installed on a client machine, though there have been reports where this issue still persists after the latest version has been installed
Timeout issues after 60 minutes - Normally resolved from increasing the timeout value from the default 60 minutes to 180 minutes for example., but sometimes the timeout can still happen even after setting this when analysing the SetupAct.log and SetupError.logs in the C:\$Windows~BT\Sources\Panther location
GPOs for Windows Update SfB Grace Period ends - After approximately a year the latest feature update is then installed on machines which can be resolved from removing the dual scan which scans for Windows Updates from the internet as well as your WSUS/SUP.

There may be an inconsistency when it comes to deploying the feature update in this method and you may want not only a more controlled method where you can deploy to a certain amounts of devices but you also want to build the consistency in successful deployments.

Benefits of Deploying a Feature Update via Application

The benefits of doing it this way are the following;

No issues of having the update show as compliant and not deploying
No timeout restriction during the actual installation
Better control of the deployment without need of using the Windows Sfb GPOs
Control on whether devices will have a forced restart or a user controlled restart

Building the Feature Update Application

Where to get the files

You can get them from two ways;

Download the Software update from the SCCM console and select a location to download it to
Download from Windows update which will end up in the C:\Windows\SoftwareDistribution\Download\

You should see two files after that;

*.ESD File

WindowsUpdateBox.exe

These two files are used to apply the Windows feature update. The *.ESD file is essentially a WIM file which is applied from using the WindowsUpdateBox.exe which we will look more into creating the install commands for this within the next section.

Once we have the files downloaded they will be needed to be placed into a folder within your content share which you would normally keep all of your applications when distributing them to your DPs.

Installation Command for the Feature Update

Install commands will follow as below along with an explanation to the commands and what more can be done with them.

Create file called "Install.cmd" and then enter the the following commands below

start /w WindowsUpdateBox /Update /PreDownload /quiet
start /w WindowsUpdateBox /Update /Install /quiet
start /w WindowsUpdateBox /Update /Finalize /quiet

The commands above take care of the intial preparation for the Windows 10 Feature update, as well as the install and the finalize step is where the machine would reboot.

If you require the update to not perform a reboot then you can add a /noreboot switch to each line.
( Note: Read the section of the deployment of the application to know how to handle the process once this switch has been added)

Steps on how to create the application

Open the Configuration Manager Console
Go to the Software Library
Right click Applications and choose Create Application
General: Specify settings for this application - Click to "Manually specify the application information" then click next.
General Information: Specify information about this application - Enter a name such as "Windows 10 Feature Update" and along with the feature update number you are looking to rollout then click next.
Software Center: Specify the Software Center entry - Click next
Deployment Types: Configure deployment types and the priority in which they will be applied for this application - Click Add.
Create Deployment Type Wizard: General: Specify Settings for this deployment type - Click to "Manually specify the deployment type information" then click next.
General Information: Specify general information for this deployment type - Provide a name for the deployment type then click next.
Content: Specify information about the content to be delivered to target devices - Browsee to the location where you have downloaded the software update and type in Install.cmd for the installation program. Then click next.
Detection Method: Specify how this deployment type is detected - Click Add Clause
Detection Rule - Configure the registry settings as below. This would be the location to get the build number ( Note: this is for 1809)

13. User Experience: Specify user experience settings for the application -
Installation Behavior: Install for System
Logon Requirement: Whether a user is logged on or not
Installation Program Visibility: Normal
Click next
14. Requirements: Specify installation requirments for this deployment type - Click next
15. Dependencies: Specify software dependencies for this deployment type - Click next.
16. Summary: Confirm the settings for this deployment type - Click next.
17. Summary: Confirm the settings for this Application - Click next.

Deploy the Windows 10 Feature Update

Now you should be ready to deploy the application to machines within a collection to perform the Windows 10 feature update.

You can also create a Task Sequence with the application so that you can initiate a restart notification, as the default of this deployment method would restart as soon as the finalize command line takes place.

SCCM: Windows 10 Feature Update showing as Compliant when not installed

Fri, 01 Nov 2019 11:03:42 GMT

Issue

When trying to deploy a Windows 10 Feature Update through SCCM onto an SCCM managed machine, you may find an issue which even though the designated machine is on a lower build number than the feature update it still shows as being compliant.

This does takes some analysis to figure out this issue. But if you right click any of the machines listed in the table you would normally expect to see the following once clicked “More Details”

But on machines which are showing a compliant message and an error code of 0x000000 for success, you will see it will show blank.

So when you investigate a machine in question you see there are no folders such as the C:\$Windows~BT which is normally created when you are installing the update, and also no trace of the update being downloaded when you look at C:\Windows\SoftwareDistribution\Download

Software Centre also shows nothing alongside all of the logs within C:\Windows\CCM\Logs such as;

· UpdatesDeployment.log

· UpdatesHandler.Log

· UpdatesStore.Log

And nothing within the C:\Windows\CCMCache folder.

In some cases you may notice that the machine has not downloaded any kind of Windows Updates or perhaps stuck on “Downloading 0%” in this case we will need to analyse the Windows Update agent and see what we get. After running a scan we are given the following error

Reason

Depending on the setup of your environment it can be down to delays within the status messages where the machine cannot get a timely response from the SUP server to actually check if the update is detected on the machine or not.

Another way to verify this is to create a Configuration Baseline with the Windows 10 Feature Update attached to it and deploy to a collection of machines that you can check this against.

If this shows as “Compliant” and the Windows 10 build is below the feature update you are rolling out then you most likely have this issue.

Workarounds

Upgrade SCCM Client to latest version

If you are a few versions down from your current SCCM version you will need to make sure that the designated machines do have the latest client version. If you are unable to set the auto upgrade option you can create a package by the client definition and deploy to a collection of machines.

1. Go to Packages

2. Right click and choose “Create Package from Definition”

3. Package Definition: Specify information about the package definition file to import – Select the Microsoft publisher and choose the Configuration Manager Client Upgrade latest version entry you see there.

4. Source Files: Specify information about the package source files – Select Always obtain source files from a source folder. Click next.

5. Source Folder: Specify the package source folder – Browse to the location of the client package which should be on your SCCM server installation path, then select the client folder.

6. Summary : Confirm the settings – Click next to finish.

Once you have created the package distribute the package to your DPs. Then deploy the package to the collection you have with the machines getting the feature updates to make sure they have the latest client version.

Then run the following actions;

· Machine Policy Retrieval & Evaluation Cycle

· Software Updates Deployment Evaluation Cycle

· Software Updates Scan Cycle

Send Status Message from client machine

With the delay of status messages we may want to send one from the client machine to manually trigger an update/response to check the compliance of the update.
( Note: Below commands referenced from reddit post https://www.reddit.com/r/SCCM/comments/80d4du/feature_update_to_windows_10_1709_compliant/ )

$SCCMUpdatesStore = New-Object -ComObject Microsoft.CCM.UpdatesStore

$SCCMUpdatesStore.RefreshServerComplianceState()

New-EventLog -LogName Application -Source SyncStateScript -ErrorAction SilentlyContinue
Write-EventLog -LogName Application -Source SyncStateScript -EventId 555 -EntryType Information -Message "Sync state ran successfully."

Once ran you can go to the Event viewer to check if the status message had been sent successfully. Once confirmed you can run the following actions;

· Machine Policy Retrieval & Evaluation Cycle

· Software Updates Deployment Evaluation Cycle

· Software Updates Scan Cycle

Alternative Feature Update Deployment

You can deploy the Windows 10 feature update differently if you are still experiencing issues in this method, which involves creating the Windows 10 feature update into a package with a program.

When the feature update is downloaded you will normally see two files

· ESD File

· WindowsUpdateBox.exe

These can be found in the following locations;

· C:\Windows\SoftwareDistribution\Download

· C:\Windows\CCMcache

· Folder which you designated to be the Software Update Package where the feature update would be downloaded to

Create Package from Windows 10 Feature Update

Once you have these files copy them to your SCCM source folders where you will create a package from. In order to install it you will need to configure a .CMD file i.e. Install.CMD.

There are three commands which are used in order to install the feature update which will explain in further detail;

· Start /W WindowsUpdateBox /Update /PreDownload – This command does the checking and preparing of your machine before the actual update is applied and also downloads the necessary files it needs.

· Start /W WindowsUpdateBox /Update /Install – This command kicks off the actual installation and this is where you start to see the C:\$Windows~BT folder.

· Start /W WindowsUpdateBox /Update /Finalize – This command prepares the machine after the installation and then performs an automatic restart. ( Note: You can stick with the first two if you don’t want to prompt for a restart right after or configure a step to show a warning before hand)

Add the commands to the Install.CMD file then place into the folder which will be used as the package.

1. Go to Packages

2. Right click and select Create New Package

3. Package: Specify Information about this package – Provide name and details and source folder where you have the feature update files. Click next.

4. Program Type: Choose the program type that you want to create – Select Standard Program. Click next.

5. Standard Program: Specify information about this standard program
Name: Install Windows 10 Feature Update (Build Number)
Command Line: Install.cmd
Program Can Run: Whether or not a user is logged on

6. Requirements: Specify the requirements for this standard program – Click next.

7. Summary: Confirm the settings – Click next to create the package.

Distribute the package and then push the program to the machines you want to upgrade.

Self-Motivation: Anyone can do IT, if I did IT - Out now to purchase

Mon, 21 Oct 2019 08:52:41 GMT

Debuted as Amazon best sellers in Paperback and Kindle

My eagerly anticipated release of my book is now available.

"Self-Motivation: Anyone can do IT, if i did IT" - is a book which is based from my experiences from when i first started out wanting to pursue a career within the IT industry.

Cannot believe that within its first day it was actually top 10 within the best sellers list in the Computer Scientist Biographies category.

Never really had an expectation for the book other than to just get my story out there to help and hopefully inspire others.

The book can be purchased online in both Paperback and Kindle in the available online stores

Upgraded to SCCM 1906: Client Registration Errors

Tue, 08 Oct 2019 14:47:04 GMT

Issue

Once upgraded to SCCM 1906, when building a machine in SCCM for whatever reason you may choose to delete the device from the system or may have machines that contain multiple network adapters which you may want it to register with a different physical/IP address.

Once you delete the machine from SCCM and try to re-register back you may encounter issues.

Investigations within the logs primarily the ClientIDManagerStartup.log located in the C:\Windows\CCM\Logs folder of your SCCM client.

Reason

The client machine normally checks to see if it is already registered within SCCM when it does exist within the console, but in this case after you have deleted the machine and can no longer be seen in the database you may still see messages like this.

The file in question which is normally used to see if the client is already registered is the C:\Windows\SMSCFG.ini file which contains all of the unique SIDs specifically for that client machine.

You can use a query such as select * from vsms_r_system where name 0 = ‘’ to check that your device is gone from the database and you will see it’s not there, but since the upgrade the functionality where the sync between the client and SCCM seems to have broken.

Solution

When upgraded to SCCM 1906, in your SCCM console in the Administration tab under the Updates and servicing you will see a hotfix for Confguration Manager 1906 Hotfix Rollup (KB4517869)

This Hotfix actually details the exact issue described within the support information.

I recommend installing this hotfix as soon as possible to avoid having this issue once you have completed your upgrade to 1906.

Optional Workaround

If you are not in a position to install the hotfix as of right now or experience this issue in other ways there is a workaround for this.

1. Delete the C:\Windows\SMSCFG.ini file

2. Reboot the machine

3. Open the Configuration Manager Applet in the control panel – System and Security

4. Go to the actions tab and run the following policies
Discovery Data Collection Cycle
Hardware Inventory Cycle
Machine Policy Cycle

Approximately after 15 – 30 minutes the C:\Windows\SMSCFG.ini file will regenerate and looking at the C:\Windows\CCM\Logs\ClientIDManagerStartup.log should start to see lines of a new registration.