Quantcast
Channel: EPMFramework Discussions Rss Feed

New Post: Option for temporary excluding a policy

0
0
I've added some functionality to EPM. We are using EPM to monitor several instances at our customers. However, every customer has its own habits for example no backup taken in the weekend. This results in a policy error on monday morning ("No recent Full backup")
So I would like to have an option to (temporary) exclude a policy. This resulted in 3 extra objects:
  • a table called policy.exclusions
  • a function to select the exclusions
  • a stored procedure which removes the policies selected by this function
In order to run this properly, run this stored procedure directly after evaluating the policies and before alerting somebody.

Table:
/*
    Table:      policy.exclusions
    Purpose:    contains information on when a policiy should be ignored
    Author:     wilfred van Dijk (wfvdijk@live.nl)
    
    Example: ignore full backups on monday
    - Policy: "Check Full Backup" 
    > insert into policy.exclusions(EvaluatedPolicy, dayofweek) values("Check Full Backup",4)
*/
use [MDW]
go

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [policy].[Exclusions](
    [EvaluatedServer] [nvarchar](50) NULL,
    [EvaluatedObject] [sysname] NULL,
    [EvaluatedPolicy] [sysname] NULL,
    [StartDate] [date] NULL,
    [EndDate] [date] NULL,
    [StartTime] [time](0) NULL,
    [EndTime] [time](0) NULL,
    [DayOfWeek] [int] NULL,
    [CategoryName] [sysname] NULL,
    [Reason] [nvarchar](64) NULL
) ON [PRIMARY]
GO
Function
/*
    function:   policy.udf_currentexclusions
    Purpose:    returns policies which can be ignored
    Author:     wilfred van Dijk (wfvdijk@live.nl)
    
    Note:       non-null values are combined and interpreted as AND
*/
use [MDW]
go

set quoted_identifier on
go

--#region drop if exists
if exists (select 1 from INFORMATION_SCHEMA.ROUTINES where ROUTINE_NAME = 'udf_CurrentExclusions' and ROUTINE_SCHEMA='policy' and ROUTINE_TYPE='FUNCTION')
    drop function [policy].udf_CurrentExclusions;
if exists (select 1 from INFORMATION_SCHEMA.ROUTINES where ROUTINE_NAME = 'udf_CurrentExclusions' and ROUTINE_SCHEMA='dbo' and ROUTINE_TYPE = 'FUNCTION')
    drop function [dbo].udf_CurrentExclusions;
go
--#endregion

create function policy.udf_CurrentExclusions(@EvaluatedServer nvarchar(50), @EvaluatedPolicy sysname, @EvaluationDateTime smalldatetime, @CategoryName sysname, @EvaluatedObject nvarchar(256))
/*
    sunday = 2^1, monday = 2^2 etc
*/
returns table 
as
    return ( 
                select *
                from policy.Exclusions
                where (EvaluatedServer = @EvaluatedServer or EvaluatedServer is null)
                and (EvaluatedPolicy = @EvaluatedPolicy or EvaluatedPolicy is null)
                and (CategoryName = @CategoryName or CategoryName is null)
                and (charindex(EvaluatedObject, @EvaluatedObject) > 0 or EvaluatedObject is null)
                and cast(@EvaluationDateTime as date) between coalesce(startdate, cast(current_timestamp as date)) and coalesce(enddate, cast(current_timestamp as date))  
                and cast(@EvaluationDateTime as time(0)) between coalesce(starttime, '00:00:00') and coalesce(endtime, '23:59:59')
                and (dayofweek & power(2,datepart(weekday, current_timestamp)) = power(2,datepart(weekday, current_timestamp)) or dayofweek is null)
    )
go
procedure:
/*
    function:   policy.usp_removeexclusions
    Purpose:    deletes policies which can be ignored
    Author:     wilfred van Dijk (wfvdijk@live.nl)
    Parameter:  @RemoveNoTargets {Y|N} removes policies with "no target" marked
                @Execute {Y|N} Deletes policies (Y) or just shows the rows (N)
                
*/
use [MDW]
go
set quoted_identifier on
go

--#region drop if exists
if exists (select 1 from INFORMATION_SCHEMA.ROUTINES where ROUTINE_NAME = 'usp_RemoveExclusions' and ROUTINE_SCHEMA='policy' and ROUTINE_TYPE='PROCEDURE')
    drop procedure policy.usp_RemoveExclusions;
if exists (select 1 from INFORMATION_SCHEMA.ROUTINES where ROUTINE_NAME = 'usp_RemoveExclusions' and ROUTINE_SCHEMA='dbo' and ROUTINE_TYPE='PROCEDURE')
    drop procedure [dbo].usp_RemoveExclusions;
go
--#endregion


create procedure policy.usp_RemoveExclusions @RemoveNoTargets char(1) = 'Y', @Execute char(1) = 'Y'
as
    begin

        set nocount on
        declare @teller int = 0

        if @Execute = 'Y' 
            begin
                delete from [MDW].[policy].PolicyHistory
                where PolicyHistoryID in (
                    SELECT distinct (a.PolicyHistoryID)
                    FROM [MDW].[policy].PolicyHistory a
                    cross apply policy.udf_CurrentExclusions(a.EvaluatedServer, a.EvaluatedPolicy, a.EvaluationDateTime, NULL, NULL) b
                )
                set @teller += @@ROWCOUNT
            end
        else
            begin
                select * from [MDW].[policy].PolicyHistory
                where PolicyHistoryID in (
                    SELECT distinct (a.PolicyHistoryID)
                    FROM [MDW].[policy].PolicyHistory a
                    cross apply policy.udf_CurrentExclusions(a.EvaluatedServer, a.EvaluatedPolicy, a.EvaluationDateTime, NULL, NULL) b
                )
            end

        if @Execute = 'Y' 
            begin
                delete from [MDW].[policy].PolicyHistoryDetail
                where PolicyHistoryDetailID in (
                    SELECT distinct (a.PolicyHistoryDetailID)
                    FROM [MDW].[policy].PolicyHistoryDetail a
                    cross apply policy.udf_CurrentExclusions(a.EvaluatedServer, a.EvaluatedPolicy, a.EvaluationDateTime, a.CategoryName, a.EvaluatedObject) b
                )
                set @teller += @@ROWCOUNT
            end
        else
            begin
                select * from [MDW].[policy].PolicyHistoryDetail
                where PolicyHistoryDetailID in (
                    SELECT distinct (a.PolicyHistoryDetailID)
                    FROM [MDW].[policy].PolicyHistoryDetail a
                    cross apply policy.udf_CurrentExclusions(a.EvaluatedServer, a.EvaluatedPolicy, a.EvaluationDateTime, a.CategoryName, a.EvaluatedObject) b
                )
            end

        if @Execute = 'Y' 
            begin
                delete from [MDW].[policy].EvaluationErrorHistory
                where ErrorHistoryID in (
                    SELECT distinct (a.ErrorHistoryID)
                    FROM [MDW].[policy].EvaluationErrorHistory a
                    cross apply policy.udf_CurrentExclusions(a.EvaluatedServer, a.EvaluatedPolicy, a.EvaluationDateTime, NULL, NULL) b
                )
                set @teller += @@ROWCOUNT
            end
        else
            begin
                select * from [MDW].[policy].EvaluationErrorHistory
                where ErrorHistoryID in (
                    SELECT distinct (a.ErrorHistoryID)
                    FROM [MDW].[policy].EvaluationErrorHistory a
                    cross apply policy.udf_CurrentExclusions(a.EvaluatedServer, a.EvaluatedPolicy, a.EvaluationDateTime, NULL, NULL) b
                )
            end
            
        if @RemoveNoTargets = 'Y' and @execute = 'Y' 
        
            begin
            
                delete from MDW.policy.PolicyHistoryDetail
                where EvaluatedObject = 'No Targets Found' 
                and PolicyResult = 'PASS';
                set @teller += @@ROWCOUNT
                
            end

        if @Execute = 'Y' 
            begin
                delete from Policy.PolicyHistory
                where PolicyHistoryID not in (
                    select distinct(PolicyHistoryId)
                    from Policy.PolicyHistoryDetail
                )
                set @teller += @@ROWCOUNT
            end
        else
            begin
                select * from Policy.PolicyHistory
                where PolicyHistoryID not in (
                    select distinct(PolicyHistoryId)
                    from Policy.PolicyHistoryDetail
                )
            end
            
        print '-- Total rows deleted: ' + cast(@teller as varchar)

    end
go

New Post: Newest version

0
0
Getting this error when selecting from view v_EvaluationErrorHistory_LastEvaluation: Msg 536, Level 16, State 4, Line 1
Invalid length parameter passed to the RIGHT function.

New Post: Newest version

New Post: Cannot find an overload for "PolicyStore"-on sql server 2016

0
0
Hi,

I'm testing framework on sql server 2016 and windows 8.1 platform. When I executed the powershell script,I got an error message like below.
.\EPM_EnterpriseEvaluation_412.ps1 -ConfigurationGroup "EPBM_TEST" -PolicyCategoryFilter "Database" –EvalMode “Check”


new-object : Cannot find an overload for "PolicyStore" and the argument count:"1".
At C:\epbm\install\ps\EPM_EnterpriseEvaluation_412.ps1:196 char:16
  • $PolicyStore = new-object Microsoft.SqlServer.Management.DMF.PolicyStore($conn)
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~
    • CategoryInfo : InvalidOperation: (:) [New-Object], MethodException
    • FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.Power
      Shell.Commands.NewObjectCommand
I created server group on CMS called EPBM_TEST and new policy category called database.
Is this issue related with the versions that I'm using?

regards

New Post: Unable to open PolicyReports.sln file to perform the configuration

0
0
Hi All,

On SQL 2014 server, we have setup the EPM framework and policies
Able to evaluate the policies manually and as well via power shell
But while configuring the reporting via SQL server reporting services, Unable to open PolicyReports.sln file via Visual Studio 2013 with the error below

"The project is incompatible with the current edition"

I tried to open the file in Visual Studio 2008, 2010, 2013 and 2015 nothing helped
Please help me understand on which version of Visual studio and SQL Server data tools to be used to open the solution file and configure for reporting

Thanks in Advance :-)

New Post: Timeout of Invoke-PolicyEvaluation

0
0
Sometimes Invoke-PolicyEvaluation fails with a timeout on some servers. Is it possible to set the timeout to a bigger value?

Microsoft.SqlServer.Management.Common.ConnectionFailureException: Failed to connect to server . ---> System.Data.SqlClient.SqlException: Connection Timeout Expired. The timeout period elapsed while attempting to consume the pre-login handshake acknowledgement. This could be because the pre-login handshake failed or the server was unable to respond back in time. The duration spent while attempting to connect to this server was - [Pre-Login] initialization=18583; handshake=14720

New Post: Cannot find an overload for "PolicyStore"-on sql server 2016

0
0
This error indicates the PowerShell can't connect to your defined variable for PolicyStore in your EPM_EnterpriseEvaluation_412.ps1 file. If you open the file you will see that you need to define the server where the output database exists along with the policies that will be used for evaluation. Your command is running against your EPBM_TEST CMS group and is filtering by the Policy Category Database that you must have defined for your policies. Hope this helps.

New Post: Stopped working after a side by side of SQL 2016 install

0
0
This was odd the job didn't error it just ran and never put anything new in the tables. I was wondering why is was green all the time.

This is the line that when I run it manually it fussed about a null value
Invoke-PolicyEvaluation -Policy $Policy -TargetServerName $ServerName -AdHocPolicyEvaluationMode $EvalMode -OutputXML > $OutputFile

I can't find any null value.

New Post: Removed servers still showing up on report

0
0
I see that the view that is using is called v_PolicyHistory_LastEvaluation and I have servers in the report/view from 6-20-16 and the servers haven't existed in a while. Any idea how these are suppose to cleared out?

New Post: Removed servers still showing up on report

0
0
The servers are part of the data warehouse historical view which is probably why they're still showing up. I'll note this as an issue to address in future update. Thanks!

Sent from my iPhone

New Post: How can I tell if I am running the latest version of EPM

0
0
The PowerShell script that I am executing is called 'EPM_EnterpriseEvaluation_412.ps1'.
I have downloaded version 4.1.2.2 to my PC, and the PowerShell script has the same name.
How can I tell if the version of EPM that I am currently executing is 4.1.2, 4.1.2.1 or 4.1.2.2 if they all use the same name?

New Post: How can I tell if I am running the latest version of EPM

0
0
Would have to check with Pedro but I believe the latest version published, although same name, should be correct version. There was minor fix he released which is why file name didn't change.

Sent from my iPhone

New Post: How can I tell if I am running the latest version of EPM

0
0
So there is no way of knowing if the version that I currently have deployed on my server is the latest version?

New Post: How can I tell if I am running the latest version of EPM

0
0
You could use a utility like beyond compare to compare file contents

Sent from my iPhone

New Post: Removed servers still showing up on report

0
0
I have a cleanupscript which I run daily to fix this kind of issues. In order to remove data from removed servers, run this:
delete
from [policy].[PolicyHistory]
where evaluatedServer not in 
(
    SELECT server_name
    FROM [policy].[pfn_ServerGroupInstances]('')
)
NOTE: policyhistorydetail will be deleted automatically because of a foreign key constraint. This may generate a lot of logging at first run.
Make sure you delete the errorhistory also:
delete
from [policy].[EvaluationErrorHistory]
where evaluatedServer not in 
(
    SELECT server_name
    FROM [policy].[pfn_ServerGroupInstances]('')

)

New Post: Poor performance with new Cardinality Estimator

0
0
I noticed a poor performance with the new cardinality estimator. In order to fix this you can either:
  • switch to compatibilitylevel 120 for your database (MSSQL 2014)
  • enable database scoped values to switch to the old estimator (MSSQL 2016)
The stacked view v_policyhistory_lastevaluation is also hammering the performance, I added the MaxDOP option for this (on a server with a lot of processors).

Anybody who has hit the same issues?

New Post: [SOLVED]: Invoke-PolicyEvaluation fails with 'Value cannot be null. Parameter name: policy'

0
0

SOLVED

I made a few changes in order to avoid the PowerShell's Invoke-PolicyEvaluation cmdlet fail. Hope it's as usefull as it was to me.
# Evaluate specific Policies against a Server List
# Uses the Invoke-PolicyEvaluation Cmdlet

[CmdletBinding()]
param([string]$ConfigurationGroup=$(Throw `
    "Paramater missing: -ConfigurationGroup ConfigGroup"),`
[string]$PolicyCategoryFilter=$(Throw "Parameter missing: `
    -PolicyCategoryFilter Category"), `
[string]$EvalMode=$(Throw "Parameter missing: -EvalMode EvalMode"))

$ErrorActionPreference = 'Stop'

# Declare variables to define the central warehouse
# in which to write the output, store the policies
$CentralManagementServer = "CONTOSO\SQL"
$HistoryDatabase = "ReportPolicyManagement"
$PolicyCategories = $PolicyCategoryFilter.Split(",");
# Define the location to write the results of the
# policy evaluation.  Delete any files in the directory.
$ResultDir = "D:\PolicyManagement\XML\"
$ResultDirDel = $ResultDir + "*.xml"
Remove-Item -Path $ResultDirDel
# End of variables

#Function to insert policy evaluation results
#into SQL Server - table policy.PolicyHistory
function PolicyHistoryInsert($sqlServerVariable, $sqlDatabaseVariable, $EvaluatedServer, $EvaluatedPolicy, $EvaluationResults) 
{
   &{
    $sqlQueryText = "INSERT INTO policy.PolicyHistory (EvaluatedServer, EvaluatedPolicy, EvaluationResults) VALUES(N'$EvaluatedServer', N'$EvaluatedPolicy', N'$EvaluationResults')"
    Invoke-Sqlcmd -ServerInstance $sqlServerVariable -Database $sqlDatabaseVariable -Query $sqlQueryText -ErrorAction Stop
    }
    trap
    {
      $ExceptionText = $_.Exception.Message -replace "'", "" 
    }
}

#Function to insert policy evaluation errors 
#into SQL Server - table policy.EvaluationErrorHistory
function PolicyErrorInsert($sqlServerVariable, $sqlDatabaseVariable, $EvaluatedServer, $EvaluatedPolicy, $EvaluationResultsEscape) 
{
    &{
    $sqlQueryText = "INSERT INTO policy.EvaluationErrorHistory (EvaluatedServer, EvaluatedPolicy, EvaluationResults) VALUES(N'$EvaluatedServer', N'$EvaluatedPolicy', N'$EvaluationResultsEscape')"
    Invoke-Sqlcmd -ServerInstance $sqlServerVariable -Database $sqlDatabaseVariable -Query $sqlQueryText -ErrorAction Stop
    }
    trap
    {
      $ExceptionText = $_.Exception.Message -replace "'", "" 
    }
}

#Connection to the policy store
#$conn = new-object Microsoft.SQlServer.Management.Sdk.Sfc.SqlStoreConnection("server=$CentralManagementServer;Trusted_Connection=true");
#$PolicyStore = new-object Microsoft.SqlServer.Management.DMF.PolicyStore($conn);
$conn = "SQLSERVER:\SQLPolicy\$CentralManagementServer\Policies"
$PolicyStore = Get-ChildItem $conn -Force | Where-Object {$_.PolicyCategory -in $PolicyCategories}

# Create recordset of servers to evaluate
$sconn = new-object System.Data.SqlClient.SqlConnection("server=$CentralManagementServer;Trusted_Connection=true");
$q = "SELECT DISTINCT server_name FROM $HistoryDatabase.[policy].[pfn_ServerGroupInstances]('$ConfigurationGroup');"

$sconn.Open()
$cmd = new-object System.Data.SqlClient.SqlCommand ($q, $sconn);
$cmd.CommandTimeout = 0;
$dr = $cmd.ExecuteReader();

# Loop through the servers and then loop through
# the policies.  For each server and policy,
# call cmdlet to evaluate policy on server

while ($dr.Read()) { 
    $ServerName = $dr.GetValue(0);
    foreach ($Policy in $PolicyStore)
   {
        &{
            $OutputFile = $ResultDir + ("{0}_{1}.xml" -f (Encode-SqlName $ServerName ), (Encode-SqlName $Policy.Name));
            #Invoke-PolicyEvaluation -Policy $Policy -TargetServerName $ServerName -AdHocPolicyEvaluationMode $EvalMode -OutputXML > $OutputFile;
            $Policy | Invoke-PolicyEvaluation -TargetServerName $ServerName -AdHocPolicyEvaluationMode $EvalMode -OutputXML > $OutputFile;
            $PolicyResult = Get-Content $OutputFile -encoding UTF8;
            $PolicyResult = $PolicyResult -replace "'", "" 
            PolicyHistoryInsert $CentralManagementServer $HistoryDatabase $ServerName $Policy.Name $PolicyResult;
        }
            trap [Exception]
            { 
                  $ExceptionText = $_.Exception.Message -replace "'", "" 
                  $ExceptionMessage = $_.Exception.GetType().FullName + ", " + $ExceptionText
                  PolicyErrorInsert $CentralManagementServer $HistoryDatabase $ServerName $Policy.Name $ExceptionMessage;
                  continue;   
            }
   } 
 }

$dr.Close()
$sconn.Close()

#Shred the XML results to PolicyHistoryDetails
Invoke-Sqlcmd `
    -ServerInstance $CentralManagementServer `
    -Database $HistoryDatabase `
    -Query "exec policy.epm_LoadPolicyHistoryDetail" `
    -ErrorAction Stop `
    -QueryTimeout 7200

New Post: [SOLVED]: Invoke-PolicyEvaluation fails with 'Value cannot be null. Parameter name: policy'

0
0
Perfect! Works for me with this code. I added back the code to delete the xml files as for some reason it wasn't cleaning up all of the time.
I was playing around with this and found that without any filter on the PolicyStore it would not pull anything back.
I commented out the where-object portion and it worked perfectly.

New Post: Problem System.ArgumentNullException, Value cannot be null. Parameter name: policy

0
0
When i run this job "Execute EPM Policies - Weekly - Maintenance - Mode "Check""
only write in the table "System.ArgumentNullException, Value cannot be null.
Parameter name: policy" in table [policy].[EvaluationErrorHistory]
Any have a solution

New Post: CodePlex shutting down - Plans for EPM

0
0
Hi,

I may be a little behind the curve here, but I have just seen that CodePlex will no longer exist next year.
I have had a look on GitHub but I cannot see Enterprise Policy Management on there.
Are there any plans to migrate this excellent project to another open source repository?




Latest Images