Educating the world

Our blog has over 10,000 readers a month

BeanCreationException NoClassDefFoundError org/hibernate/usertype/UserType during Tomcat start up

July 6th, 2011

I have just spent the last 2 days looking for why my Spring container (under Tomcat) was getting ClassNotFoundException and NoClassDefFoundError during start up. There is so much mis-leading stuff on the Internet.

I had 2 projects in Eclipse: the core application project, that depends on, a different project containing all the hibernate and dao managers. Both projects’ JUnit tests work without any problems, but when I tried to launch the Tomcat container it failed with the following error:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name ’sessionFactory’ defined in class path resource [uk/co/bigsoft/proj/database.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/hibernate/usertype/UserType

I could confirm that the org.hibernate.usertype.UserType was in my Web Apps Libraries folder and was accessible by my application.

I was on the verge of suicide when I found an article which provided a vital clue.

The problem is caused by classloader dependencies - that is, the scope of jar files which are accessible by the application at a particular point. This point being during the Tomcat startup (bootstrapping phase).

Although, I hit the problem with Hibernate it could happen with any User Entry jar.

For completeness here was the full exception:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [uk/co/bigsoft/sm/config/database.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/hibernate/usertype/UserType

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1362)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:540)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:485)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:251)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:169)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:248)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:170)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:407)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:735)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:369)
	at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:251)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:190)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4350)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
	at org.apache.catalina.core.StandardService.start(StandardService.java:516)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.lang.NoClassDefFoundError: org/hibernate/usertype/UserType
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
	at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1275)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1206)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:169)
	at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:123)
	at org.hibernate.type.TypeFactory.heuristicType(TypeFactory.java:246)
	at org.hibernate.mapping.SimpleValue.getType(SimpleValue.java:283)
	at org.hibernate.cfg.Configuration$1.getIdentifierType(Configuration.java:2139)
	at org.hibernate.type.EntityType.getIdentifierType(EntityType.java:520)
	at org.hibernate.type.EntityType.getIdentifierOrUniqueKeyType(EntityType.java:545)
	at org.hibernate.type.ManyToOneType.getColumnSpan(ManyToOneType.java:88)
	at org.hibernate.mapping.SimpleValue.isValid(SimpleValue.java:276)
	at org.hibernate.mapping.ToOne.isValid(ToOne.java:105)
	at org.hibernate.mapping.Property.isValid(Property.java:207)
	at org.hibernate.mapping.PersistentClass.validate(PersistentClass.java:458)
	at org.hibernate.mapping.RootClass.validate(RootClass.java:215)
	at org.hibernate.cfg.Configuration.validate(Configuration.java:1135)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1320)
	at org.springframework.orm.hibernate3.LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:753)
	at org.springframework.orm.hibernate3.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:691)
	at org.springframework.orm.hibernate3.AbstractSessionFactoryBean.afterPropertiesSet(AbstractSessionFactoryBean.java:211)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1390)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1359)
	... 29 more
Caused by: java.lang.ClassNotFoundException: org.hibernate.usertype.UserType
	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
	... 64 more
12-Nov-2008 12:04:56 org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [uk/co/bigsoft/sm/config/database.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/hibernate/usertype/UserType
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1362)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:540)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:485)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:251)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:169)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:248)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:170)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:407)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:735)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:369)
	at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:251)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:190)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4350)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
	at org.apache.catalina.core.StandardService.start(StandardService.java:516)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.lang.NoClassDefFoundError: org/hibernate/usertype/UserType
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
	at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1275)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1206)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:169)
	at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:123)
	at org.hibernate.type.TypeFactory.heuristicType(TypeFactory.java:246)
	at org.hibernate.mapping.SimpleValue.getType(SimpleValue.java:283)
	at org.hibernate.cfg.Configuration$1.getIdentifierType(Configuration.java:2139)
	at org.hibernate.type.EntityType.getIdentifierType(EntityType.java:520)
	at org.hibernate.type.EntityType.getIdentifierOrUniqueKeyType(EntityType.java:545)
	at org.hibernate.type.ManyToOneType.getColumnSpan(ManyToOneType.java:88)
	at org.hibernate.mapping.SimpleValue.isValid(SimpleValue.java:276)
	at org.hibernate.mapping.ToOne.isValid(ToOne.java:105)
	at org.hibernate.mapping.Property.isValid(Property.java:207)
	at org.hibernate.mapping.PersistentClass.validate(PersistentClass.java:458)
	at org.hibernate.mapping.RootClass.validate(RootClass.java:215)
	at org.hibernate.cfg.Configuration.validate(Configuration.java:1135)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1320)
	at org.springframework.orm.hibernate3.LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:753)
	at org.springframework.orm.hibernate3.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:691)
	at org.springframework.orm.hibernate3.AbstractSessionFactoryBean.afterPropertiesSet(AbstractSessionFactoryBean.java:211)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1390)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1359)
	... 29 more
Caused by: java.lang.ClassNotFoundException: org.hibernate.usertype.UserType
	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
	... 64 more

Replying on the same email address you receive email on with GMail

July 4th, 2011

Working in I.T. means that I have several email addresses or more specifically I receive email sent to me on several email addresses.

When Acme Corporation asks me to build their network I create the email address acme@example.com for them to use. Everyone at Acme uses that email address to get in contact with me. When I have to talk to their prospective broadband provider I use that email address. In cases when I have control of their email system I create a generic I.T. person email address on their domain like it@acme.com and use that. When we part company I just reassign that address to one of their people and the handover is seamless.

When I receive email on a particular address I would like to respond on that email address. This helps me to separate the work. I create an email address for a client, then use that address for everything I do with them. Sure, you could use your normal email address but the more your email address is used in dealing with 3rd party companies in relation to your client company, the more difficult it is to remove it when the time comes. You will find that in spite of what those 3rd party companies have said, you are registered on all sorts of mailing lists and you’ll forever be receiving sales promotions about broadband services that you are not interested in any more.

When you come to parting ways the owness is on you to un-register yourself from all these lists. All email will be bounced back to the sender and you don’t have to do anything. It is far easier to just delete the email address then the owness is on the 3rd party companies to do the chasing.

  1. Login to GMail’s web interface.
  2. Click Settings in the top right.
  3. Click the Accounts tab.
  4. In the Send mail as section, click the link Add another email address you own.
  5. A window will pop-up and ask you to email an email address.
  6. Enter the email address you would like to send email as: it@acme.com and click Next Step.
  7. To combat spam GMail must send a verification email to the address you specify to make sure you are capable of receiving email on that address.
  8. So click the Send Verification button.
  9. The window moves on to a screen that lets you enter a confirmation code which is in the email GMail has just sent you.
  10. Depending on lots of complicated stuff the email might arrive in 1 second or it might arrive tomorrow, so I close this window down but clicking the Close window link at the bottom.
  11. Check your inbox periodically and when the email arrives it will have a subject of Example Confirmation - Send Mail as it@acme.com.
  12. Open up the email and click the confirmation URL embedded in the email.
  13. You will be taken to a web page that confirms that you can now send email as this new user.
  14. Head back into the Settings->Accounts section of GMail.
  15. In the Send mail as section, make sure When replying to a message is set to Reply from the same address to which the message was sent.

Scripting with Windows PowerShell Part 4

July 3rd, 2011

Working with Windows PowerShell to retrieve information from remote computers

I’ve kind of got used to Ed Wilson’s chaotic style of presenting. He must be clever to achieve such a position inside Microsoft, but each of his presentations looks and feels as though he has done no preparation what so ever. It really comes over as off the cuff with all the problems associated with that. Here we are in part 4; in real time this is the fourth consecutive day he has presented and again it really feels like he hasn’t bothered with his practice run.

Half of the first 5 minutes were devoted to him rambling about the scripting games that his wife had just signed up for on the HeyScriptingGuy website, and the other half was watching him try to find the web page where she did it. Unfortunately he couldn’t find the page. He must have been sweating because he was jumping from page to page and hammering the page up/down buttons looking for it to no avail. The video feed struggled with the speed and so the top half of the screen was of one page and the bottom half was the scroll window from a couple of page-up’s ago. Then he started mumbling again and quickly changed the subject. What a fiasco!

After that embarrassment we get into what today’s lesson is all about.

Windows PowerShell ships with an ISE which stands for Integrated Scripting Environment or Integrated Script Editor or Incredible Script Editor. The third option is clearly a joke but which of the 2 first choices is correct Mr. Microsoft man? Surely he must know but has forgotten.

Apparently there are some other script editors out there. There was a brief stutter where he tried to say a list but just at the last second decided he couldn’t think of any off the top of his head and quickly changed the subject - nice! “But they are really cool and have a lot of feature".

The nice thing about Windows Powershell ISE is that it comes bundled for free and “also it works pretty good” - and that would seem to be it!

Just as PowerShell comes in a 32-bit version as well as the 64-bit version, so too does PowerShell ISE. Most of the time it won’t make any difference which one you use. Occasionally there will be incompatibilities with things like (some) COM commands so you will have to detect the failure and rerun the script with the correct bit version. There are a couple of HeyScriptingGuy articles on this subject.

PowerShell ISE has 3 sections. The top section is the script section which works like the command line in that you can hit the tab button to get command completion. Ed says that in his scripts, he likes to split the command up over several lines and he uses the pipe symbol as his new line point. What he doesn’t quite make clear is that you can do this in the command line as well. The command line interpreter knows when it’s the end of the statement because that’s what command line interpreters do!

You could have typed it into the third panel which is the execution pane or the command pane. Press F5 or the green triangle to run the script.

Get-Process |
Sort-Object -Property cpu |
Format-Table -Property name, cpu

The results appear in the middle pane.

PS C:\> Get-Process |
Sort-Object -Property cpu |
Format-Table -Property name, cpu

Name            CPU
----            ---
System
Idle
audiodg
svchost         0
svchost         0
atiesrxx        0
EngineServer    0
nsd             0.0156001
svchost         0.0156001
conhost         0.0156001
...

You will notice that some of the CPU values are missing, so we’ll use a filter to remove those entries. We know from previous lessons that we need the Where-Object cmdlet, but where do we add it? We can’t add it after the Format-Table because the Format-Table is just a text formatter. The text loses all its record or object information so the filter command won’t have anything to run against. We could add it after the Soft-Object but this would be a waste because we would spend time sorting entries that we are going to get rid of. Obviously we can’t do it before the Get-Process! so the best place is after the it as close to the source as possible. “Don’t forget to close the curly brackets, because it’s really easy to forget” - wow thanks Ed!

We can use the wind-screen wiper button in the tool bar to clear Output Pane. All that back and forth with the mouse was driving is a bit amateur-ish, I found it much easier to control+d to jump to the execution pane type cls and hit return, then control+i; that way your hands never leave the keyboard and it’s a bit more fluid.

Name        CPU
----        ---
nsd         0.0156001
conhost     0.0156001
svchost     0.0156001
wuauclt     0.0312002
lockmgr     0.0312002

Ed said “We’ve only gotten the CPU information from the processes that are reporting that information” and then moves on. I think it’s important to notice that that is not what has been returned. You will notice that all the processes that reported zero CPU are missing too. His $_.cpu filter is a standard true/false condition so empty and zero are both false and this will skew your results. A better condition would have been:

Where-Object { $_.cpu -or $_.cpu -eq 0 }

This condition says keep everything that is not blank or zero or everything that is zero. It’s a bit convoluted but it could be seen as similar to the standard test for empty in Java:

var != null && “".equals(var)

We can change the order of the list by placing the caret immediately before the pipe symbol on the Sort-Object line and typing -D then hitting the tab key to complete the word Descending, then hit F5 to run the new statement. This will return the list in reverse order.

In my representations of the output I have removed most of the white space so it will fit on the page without wrapping but you can do this as part of the Format-Table cmdlet by adding the auto size switch:

Format-Table -Property name, cpu -AutoSize

One thing to note is that the auto size option can add a lot of extra running time to the cmdlet as it has to read in all the results before it can start displaying any. Only once all the results are held can the cmdlet determine the correct width for all the columns. Without the option, it can just do an offset from the width of the page and takes a best guess. For example if you were listing all the files on your hard drive it would take a while for the statement to start returning anything:

Get-ChildItem c:\ -Recurse | Format-Table length, name -AutoSize

If you don’t use auto size you will see the columns shift as it learns more about the lengths of each column.

The tab in the scripting (top) pane still reads “Untitled1.ps1*” and so it’s not considered a script yet (I think that’s open to debate: is a book a book if it’s in electronic form?) so he saves it using the blue disc icon in the tool bar and calls it SortProcessesByCPU.ps1 “and then I say groovy” (I thought we were done with that!)

Now that we have saved the script, when we try to run it “three, two, one, boom we got an error” and it writes Failed in the status bar at the bottom of the window.

PS C:\> C:\SortProcessesByCPU.ps1
File C:\SortProcessesByCPU.ps1 cannot be loaded because the execution of scripts is disabled on this system. Please see "get-help about_signing" for more details.
At line:0 char:0

“Everything’s cool. The ‘help about_signing’ is almost a little bogus, y’know, what we really need to see is help about the execution policy and you don’t have to sign your scripts if you don’t want to. In fact, in general I don’t see a whole lot of value from that.” - you heard it hear first!

We could run the command in PowerShell ISE but it only stopped us running the script when we saved it. So go to the bottom pane which he has started referring to as the immediate pane and type:

PS C:\> Get-ExecutionPolicy
Restricted

Restricted is the default (as installed).

Enabling Script Execution

In order to enable script execution on your system there are a number of things you can do.

  • Run the Set-Execution cmdlet with administrator rights which gives it permission to write to the registry.
  • You can just hack the registry but they don’t recommend doing that!
  • You can set it in the Enterprise wide Group Policy.

Enabling script execution is a security convenience and not a security feature. “The reason I say this, other than the fact it’s true is, is that there is a by-pass execution policy parameter, cos this is actually real cool and it’s a feature that was requested by a lot of our customers. They wanted the scripts to be locked down by default or secured but they still wanted the ability to run scripts". There followed an example of the difference between a convenience and a feature. So to cut a really long story short. A convenience is like the seat belt alarm in your car that reminds you to put your seat belt on. That’s a convenience to help you remember where as the actual seat belt and the air bags are the security features.
The alarm might be a bit annoying but it won’t save your life where as the other features will.
Sounds like bollox to me but there you have it.

So in summary it’s a feature that everyone asked for, that’s not a feature, that switches off all the security which doesn’t add anything anyway. Clear? Let’s see if the examples explain this any better!

We are going to open a command window and navigate to the place where we saved our SortProcessByCPU.ps1 and we are going to use the execution policy bypass to run our script:

powershell -executionpolicy bypass -noexit -file .\SortProcessesByCPU.ps1

This command will bypass the security and run the script, without exiting, and drop us on the command line running in by-pass mode. If you type:

PS C:\> Get-ExecutionPolicy
Bypass

From this newly created command line you can run scripts just by typing their name. The shell will use the extension to launch the appropriate helper application. This works the same as running it in the old command.exe. The script will use the execution policy that the shell window has, so in this case that would be Bypass.

There is another got-cha hidden away here as well. The order in which you specify the switches on the command line (for the powershel.exe command) makes a difference! So if is doesn’t work you will have to re-organise the switches until it does work!!!! To simplify, I think you should set the execution policy first so that it knows how to interpret the rest of the command line switches. Brilliant!!!

If we type Set-ExecutionPolicy into the command window it will give us the list of valid enumerations:

cmdlet Set-ExecutionPolicy at command pipeline position 1
Supply values for the following parameters:
ExecutionPolicy:
Set-ExecutionPolicy : Cannot bind parameter 'ExecutionPolicy'. Cannot convert value "" to type "Microsoft.PowerShell.Ex
ecutionPolicy" due to invalid enumeration values. Specify one of the following enumeration values and try again. The po
ssible enumeration values are "Unrestricted, RemoteSigned, AllSigned, Restricted, Default, Bypass, Undefined".
At line:1 char:20
+ Set-ExecutionPolicy <<<<
    + CategoryInfo          : InvalidArgument: (:) [Set-ExecutionPolicy], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.SetExecutionPolicyCommand

The list of valid enumerations is:

  • AllSigned - all scripts are signed with a code signing certificate. If you are using Microsoft PKI you just add a new PKI certificate type of code signature, then you can issue it out to your network administrators. This mode will not stop users running any script they like because they can just run it in Bypass mode!
  • RemoteSigned - (The important bit of Ed’s explanation for this was misleading, confusing and, after I had confirmed it in documentation, dead wrong!) So here is what it really does: scripts from an untrusted location are going to have to be signed. This works in combination with the Internet Option’s zones. There is a chance that the file servers on your network are in an untrusted zone. So you would need to use Group Policy to add those servers to the trusted zone so that they don’t need to be signed. Local file system PowerShell scripts are considered trusted so this option means that you won’t need to sign local files either.
  • Restricted - Does not load configuration files or run scripts. On my system this is the default.
  • Unrestricted - Ed sets everything to this because “it works". If you are downloading scripts from the Internet or an untrusted zone you will still get a warning about downloading PowerShell scripts, but that message comes from Internet Explorer. He said using ByPass will stop this. He then talks about downloading a modules from the Internet and when it comes down each item in the zip file is marked as from an untrusted network so it’s constantly prompting him for validation. The best way to get around this is to remove the internet from the untrusted zone - WHAT!?!?!?!? I don’t know about you, but that sounds like a *really* bad idea to me, but this is the kind of useless dangerous and misleading advice that Ed has been feeding us during these presentations.
  • Default - Not listed in the documentation, so I’m assuming the the default (Restricted) is just the default default and that the Default option is what ever the default has been set to, which may vary from system to system.
  • Bypass - Nothing is blocked and there are no warnings or prompts.
  • Undefined - Removes the currently assigned execution policy from the current scope. This parameter will not remove an execution policy that is set in a Group Policy scope.

When you try to change the policy it throws up the warning:

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic. Do you want to change the execution
policy?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"):

If you don’t have administrator rights it will fail anyway with Access to the registry is denied. You should right click on the PowerShell icon to Run As Administrator. Depending on your set up you might get the following error:

sPowerShell\Microsoft.PowerShell_profile.ps1 cannot be loaded because the execution of scripts is disabled on this system. Please see "get-help about_signing" for more details.
At line:1 char:2
+ . <<<<  'C:\Users\me\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1'
    + CategoryInfo          : NotSpecified: (:) [], PSSecurityException
    + FullyQualifiedErrorId : RuntimeException

You will only get this message if you have a PowerShell profile. The PowerShell profile is a script that gets run when you open a PowerShell command prompt. It may contain customisations like aliases or import module commands. The default location of the PowerShell profile has the nice-and-easy-to-remember path of: C:\Users\<user>\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1. The path to you profile is also held in the $profile environment variable.
If you are running with an execution policy of Restricted then you will get the scripts disabled error as it tries to run your profile.

From our administrator PowerShell command window we are going to set the execution policy to Unrestricted. Do you remember that totally unrelated problem in the event log lesson that filtered the list of logs and “using the force” made it work. So Ed starts saying “use the force” again, so that the Set-ExecutionPolicy won’t prompt us for confirmation. How are they related other then saying “use the force". He is so irritating!

Set-ExecutionPolicy restricted -Force

The Set-ExecutionPolicy cmdlet run by the administrator updates the registry to make this change permanent. Now all new PowerShell command windows, as well as existing open PowerShell command windows will be set to the new policy of Unrestricted.

So what is a Windows PowerShell Script? It’s a bunch of Window’s PowerShell commands a bit like a batch file, “but there are language thingys that can be used as well". He explains that a language thingy is a semi-colon that can separate commands on the same line.

PS C:\> echo hi ; echo bye
hi
bye

Ed describes his way of working and I can’t believe my ears! He has PowerShell console open on one monitor and Powershell ISE open on the other. He practices commands in the PowerShell console and when it is correct he copies it into the Powershell ISE. How does he do any work at this pace? He spent most of yesterday explaining that the Powershell ISE has incorporated into it the PowerShell console so either he has forgotten or he’s a bit simple.

Jeffery Hicks came up with this easy way to create PowerShell scripts and as Ed Willson didn’t do any preparation for his tutorials he thought he’ll just pilfer it (but at least he passed on the credit).

We are going to populate the command history, read it out into a file and run it as a script. With all the lessons we have done so far you should have no problem doing this yourself.

We will start off by clearing our history with Clear-History and creating some more. The examples that Ed gave were generic and a bit useless so let’s try and come up with some that you already know that you might find more useful. So enter the following at the command prompt which will give us the top 5 processes that have the most CPU time, the 5 most recent application log entries and all the running services respectively.

  • Get-Process | Sort-Object { $_.cpu } -Descending | Select-Object -First 5 | Format-Table cpu, processname
  • Get-EventLog -LogName application -Newest 5
  • Get-Service | Where-Object { $_.status -eq "Running" }

As you can see from my Get-History I made a few typos so we’ll have to clean it up.

Id CommandLine
-- -----------
27 Get-Process
28 Get-Process | Sort-Object $_.cpu
29 Get-Process | Get-Member
30 Get-Process | Sort-Object { $_.cpu }
31 Get-Process | Sort-Object { $_.cpu }
32 Get-Process | Get-Member
33 Get-Process | Sort-Object { $_.cpu } -Descending
34 Get-Process | Sort-Object { $_.cpu } -Descending | Select-Object -First 10
35 Get-Process | Sort-Object { $_.cpu } -Descending | Select-Object -First 10 | Format-Table processname
36 Get-Process | Sort-Object { $_.cpu } -Descending | Select-Object -First 10 | Format-Table cpu, processname
37 Get-EventLog -LogName application
38 #Get-Process | Sort-Object { $_.cpu } -Descending | Select-Object -First 10 | Format-Table cpu, processname
39 Get-EventLog -LogName application -Newest 5
40 Get-Service
41 Get-Service | Get-Member
42 Get-Service | Where-Object { $_.status -eq "Running" }

If you remember from last time, each history element is an object so we can do a Get-Member to see what properties are available.

Get-History | Get-Member

   TypeName: Microsoft.PowerShell.Commands.HistoryInfo

Name               MemberType Definition
----               ---------- ----------
Clone              Method     Microsoft.PowerShell.Commands.HistoryInfo Clone()
Equals             Method     bool Equals(System.Object obj)
GetHashCode        Method     int GetHashCode()
GetType            Method     type GetType()
ToString           Method     string ToString()
CommandLine        Property   System.String CommandLine {get;}
EndExecutionTime   Property   System.DateTime EndExecutionTime {get;}
ExecutionStatus    Property   System.Management.Automation.Runspaces.PipelineState ExecutionStatus {get;}
Id                 Property   System.Int64 Id {get;}
StartExecutionTime Property   System.DateTime StartExecutionTime {get;}

There is a property called CommandLine which we will use:

Get-History | % { $_.commandline >> file.ps1 }

The above command gets the history objects and pipes them into the % which is short hand for ForEach-Object, and for each object run the script block appending the output to file.ps1. So fully expanded it would read:

Get-History | ForEach-Object { echo $_.commandline >> file.ps1 }

Personally, I don’t like their way. I think that the following way is far simpler with less typing and as it turns out better because file.ps1 will be emptied and the full history will be added whereas in their example if the file.ps1 exists it will be appended to; which is probably not what you want.

Get-History | ft commandline > file.ps1

The Jeff fellow sounds almost as good as Ed, who is Microsoft employing?

Once all the commands are in a file we can load it into Notepad or the Powershell ISE and edit out the lines we don’t want.

Questions and Answers

Q: When are you going to do the webcast on remote PowerShell?
A: Summer probably.
My A: This web cast was supposed to be about remoting but Ed elected to talk about execution policies instead.

Q: What are the best books for someone wanting to learn PowerShell?
My A: Ed obviously things that the books he wrote on PowerShell are the best, but if his books are anything like his webinars then I would steer well clear and try something else instead - like the internet!

Q: Ed mumbled though the question but I think it was something to do with how the Scripting Games work?
A: There will be a beginner event and an advanced event. Once you start competing you can’t do both. You will have 7 days to upload you solution. After 7 days the judges will grade them and post the scores. So every day there will be a leader board, so at the end of the 2 week period all the scores will be totalled and there will be prizes. There will also be daily prizes too. There is an FAQ on the site.

Q: Unfortunately he mumbled through reading the question again but it sounded like an interesting question about Importing Modules because Ed had brushed over it previously.
A: That’s going modules and he didn’t have time to get into that. Strange as there is still 10 minutes on the clock, I’m now wondering what it will hold! Go to the HeyScriptingGuy site click the tag for GettingStarted then MNodules, “but basically you use Import-Module and it will import your modules, then you can use Get-Command specifying which modules you want to look at. If you’re not sure if you are a beginner or an advanced user then you can take the PowerShell quiz.

Q: Mumble, mumble… commands similar to pause that will keep a script running, after running a ps1 file ?
A: Well, there’s Start-Sleep that will kind of halt execution of a script, so am thinking that maybe you could do that.

Q: Inside of using SRP from Windows 2008 R2, could you use the AppLocker?
A: Yes you could.

Q: I have a need to manage Windows 2003 remotely using PowerShell. Is it mandatory to use WinRM on a remote machine, is it a security issue?
A: In general I don’t know of any major issues using WinRM. WinRM is designed to be firewall friendly and it’s going to be better to use WinRM than using the other options. So you would install Windows PowerShell 2.0 on your remote machine, then you would enable remoting and at that point we could use WinRM and travel over there. If you didn’t want to do that you could use WMI to manage it remotely, but dude I mean WMI requires a whole lot more holes in your (?poorp? don’t understand what he said for this word - maybe it slang for firewall configuration?) because it’s not designed to be firewall friendly so you would really be better off using WinRM and PowerShell.

Q: I noticed that your scripts are ps1 in PowerShell 2?
A: Ha, ha, ha, yes ps1 is the file extension. This question has been asked a whole lot of times. There is no .ps2 extension. It does imply that, a erm, erm, we’re not necessarily backward compatible but we are forward compatible. So that means that things that ran in PowerShell 1 should run in PowerShell 2. Now I can write stuff in PowerShell 2 and have it run on PowerShell 1.0 if I don’t use any of the new cmdlets and any of the new features, but we made a lot of changes even to cmdlets that existed in PowerShell 1 and so it’s really, really hard to keep track of all of the changes. And erm, y’know so so in that regard y’know you probably y’know erm need to watch out. Erm I don’t know of any specific reasons why you would need to use PowerShell 1 as opposed to PowerShell 2 unless for instance you were using the SQL-Mini shell or something like that.
My A: We would have had .ps for extension but that is taken up for PostScript files so one of the Microsoft wizards decided to .ps1 would be the next best thing. Out of all the possible extensions like .powershell, .psh then picked .ps1. Once you have selected an extension then you generally never change it unless a new version won’t parse because of structural changes.

Q: I get an error when I try Unrestricted just like I followed you.
A: That’s probably because you’re PowerShell console did not have Administrator rights. So you have to grant Administrative rights, you’ve got to run as Administrator to change the script execution policy.

Q: What is the difference between Unrestricted execution policy and Bypass?
A: The difference is Unrestricted will still give you a message when you have downloaded scripts from the internet and you run it, Bypass does not.

Q: Is the dot ?flag? notation really only used in PowerShell 1. Does it have much purpose in 2.0?
A: I’ve used it in 2.0. You can use it to include a file in a script or in a console. Most of the time we’re not doing that now-a-days because everyone’s putting stuff that would be included in an include file they’re putting them into modules. It still exists and is still around.

Q: Is Force available for all modifying commands?
A: I’m not certain about that, I always a little hesitant when a read “all", I don’know, it’s on a bunch of them. I haven’t specifically done a check to see if it is on all of them or not.

Q: Is there a way to comment a ps1 script file?
A: Yes the pound sign is the comment character in PowerShell 1, it’s also the comment in PowerShell 2. We also have a block comment which is an angled bracket pound followed by an angled bracket pound that you can use in PowerShell 2.0
My A: Not sure why Ed is using a Eurpean keyboard that has a Great British pound symbol instead of a hash key. So in PowerShell 1 it’s just a single line # and in PowerShell it’s the single line hash and everything between <# … #>

Q: Would you recommend your book for a very basic beginner?
A: Lot’s or erm’s and arh’s, but basically watch these 5 videos then you’ll be ready for the book.

Q: What does dot sourcing mean?
A: Well dot sourcing is like when I say a dot space to include a file into a script or to include a script into a PowerShell console. It’s a way of causing it to run and to pick up the files and stuff that are in there and functions and stuff and to bring them into the current session.

Q: If we configured the remote execution policy to be RemoteSigned then the user can pass it anyway?
A: Yes.

Q: Is there a was to display output if a command runs successfully?
A: If you write the script in such a way that you can do that, then yes. I’ve got an article that I was working for tomorrow where the scripting wife wanted to shut-down computers and doesn’t return any information. You can check the error object $error and see if there were any errors or not or $?. And if those are there, then that would be cool.

Q: What is the best book for remote PowerShell scripting?
A: I don’t know of any book that specifically cover remote PowerShell scripting in particular. Then he shamelessly plugs his book because it has a chapter on it. Leigh Holmes’ book talks about it, but no sure if there is a whole chapter. Richard Sidaway’s book PowerShell in Practice, Don Johnson and Jeffery Hicks’ book. Any PowerShell book is going to cover it.

Zen Internet's customer portal

June 21st, 2011

For a company with such a good track record for customer support their online customer portal leaves a lot to be desired.

These are a couple of the things I need to do every now and again, but I just seem to spend ages hunting around for the options. The site is really slow and uses some sort of awful home grown Javascript front end that means you can’t open more than one window at a time otherwise the site won’t work as expected.

Show outstanding invoices:
Click My Billing
In the Quick links section click View your site’s current billing

Scripting with Windows PowerShell Part 3

June 16th, 2011

Working with Windows Management Instrumentation (WMI) information to retrieve essential information
Part 3 of a 5 part series.

Event logs

These commands will work on the local machine or a machine across the network. Most of the EventLog commands accept Computer name as an option, but do not accept any credentials. So connections made to a another machine must be able to impersonate you. If the security hasn’t been set up that way then you must use a different method such as the PowerShell Remoting commands (Invoke-Command), WMI (with Get-WmiObject) or Enter-PSSession.

Get-EventLog -List

Lists all the classic event logs. Classic seems to refer to the log names that were around years ago in NT3/NT4 days, but this list contains classic and new logs. So Ed Wilson, we’re just refer to this list of logs as a list of logs - ya’know to avoid any confusion!

Max(K) Retain OverflowAction        Entries Log
------ ------ --------------        ------- ---
   512      7 OverwriteOlder            724 ACEEventLog
20,480      0 OverwriteAsNeeded       4,146 Application
20,480      0 OverwriteAsNeeded           0 HardwareEvents
   512      7 OverwriteOlder              0 Internet Explorer
20,480      0 OverwriteAsNeeded           0 Key Management Service
 8,192      0 OverwriteAsNeeded           0 Media Center
   128      0 OverwriteAsNeeded          11 OAlerts
20,480      0 OverwriteAsNeeded       4,621 Security
20,480      0 OverwriteAsNeeded       7,618 System
15,360      0 OverwriteAsNeeded         231 Windows PowerShell
...

The “Windows PowerShell” (above) log is pretty useless and will probably just have a load of messages saying “powershell started". We can read a log by using its log name on the command line:

Get-EventLog -LogName application -Newest 5

Index Time          EntryType   Source                 InstanceID Message
----- ----          ---------   ------                 ---------- -------
 4146 Jun 03 16:23  Information Office Software P...   1073742827 The Software Protection service has completed
 4145 Jun 03 12:06  Information McLogEvent             1073746824 McShield service started....
 4144 Jun 03 11:05  Information Windows Error Rep...         1001 Fault bucket , type 0...
 4143 Jun 03 11:05  Information McLogEvent             1073742081 The scan of D:\DOWNLO~1\ECLIPS~2.ZIP has taken
 4142 Jun 03 11:05  Information McLogEvent             1073742081 The scan of D:\DOWNLO~1\ECLIPS~1.ZIP has taken

Each one of the entries has 6 columns, but as we know from previous lessons these 6 columns are just a cut down version of what is available. To see more we’ll use the Format-List to show them all:

Get-EventLog -LogName application -Newest 1 | Format-List *

EventID            : 1003
MachineName        : cuckmere.example.com
Data               : {}
Index              : 4146
Category           : (0)
CategoryNumber     : 0
EntryType          : Information
Message            : The Software Protection service has completed licensing status check.
                     Application Id=59a52881-a989-479d-af46-f275c6370663
InstanceId         : 1073742827
TimeGenerated      : 03/06/2011 16:23:19
TimeWritten        : 03/06/2011 16:23:19
UserName           :
Site               :
Container          :

It might be nice to see on what other occasions The Software Protection service ran.

Get-EventLog -LogName application | Where { $_.message -match ‘protection’ }

The -match takes a regular expressions so in our case it is looking for the word “protection” anywhere on the line. The results come back and you may have to take advantage of the control+c keyboard sequence to stop the current command.

Index Time          EntryType   Source                 InstanceID Message
----- ----          ---------   ------                 ---------- -------
 4146 Jun 03 16:23  Information Office Software P...   1073742827 The Software Protection service has comple
 4116 May 31 14:28  Information Office Software P...   1073742827 The Software Protection service has comple
 4115 May 31 14:28  0           Office Software P...   1073742726 The Software Protection service has starte
 4114 May 31 14:28  Information Office Software P...   1073742890 Initialization status for service objects.
 4113 May 31 14:28  Information Office Software P...   1073742724 The Software Protection service is startin
...

Ed tried to make excuses for the slowness of his results being displayed. “But that’s cool because we’re filtering out through a whole lot of entries, then doing a match of the message, and it’s like having to read all the messages and look for that sequence of characters.” - Really? I mean you would expect a little delay between the results being displayed, but not that much. It looks to me like he hadn’t installed the graphics card on his PC properly and the scroll rate had dropped through the floor - that old chestnut &amp;#59;&amp;#41; Anyway I tried it and it was fine. With event logs we can filter on InstanceID by adding the -InstanceId switch.

Get-EventLog -LogName application -InstanceId 1073742827

While the results are equivalent to a statement containing a Where-Object cmdlet, it is much quicker because the instance filtering is done inside the Get-EventLog cmdlet i.e. the non-matching InstanceId records never have to travel down the pipe. Wow, that’s another 5 minutes I’ll never get back.

Get-EventLog -LogName application |where {$_.InstanceId -eq 1073742827}

Let’s say we want to filter on Source. We can’t see all of the Source name so we can expand that column using the Select-Object cmdlet (Select is an alias for Select-Object):

Get-EventLog -LogName application -InstanceId 1073742827 -Newest 5 | select source

Source
------
Software Protection Platform Service
Windows Search Service
Office Software Protection Platform Service
Office Software Protection Platform Service
Software Protection Platform Service

All values of the source column have been fully expanded. For the purpose of expanding column names we could have just used one of the format cmdlets (Format-Table / Format-List). Select-Object is supposed to manipulate lists of objects such as finding unique entries or returning a sub-list. So to finish the previous example, we can specify the full source name or we can use a wild card:

Get-EventLog -LogName application -Source “Software Protection Platform Service” Get-EventLog -LogName application -Source “Software Protec*”

In the past Ed’s “always hated typing dates, erm… simply for the fact, y’know, you never really know exactly how these things are going to be interpreted, and all of that", but not with PowerShell because there are filter switches of -Before and -After. Perhaps he should revisit the other search applications and read the instructions! The -Before and -After switches expect dates so the format is pretty flexible and is based on your locale or as they call it Culture. To find out which Culture you are in, issue the command:

Get-Culture

LCID             Name             DisplayName
----             ----             -----------
2057             en-GB            English (United Kingdom)

Getting back to our date filtering example. An interesting shortcut is that -A is short for -After, -B is short for -Before and as it turns out you can shorten a switch to it’s minimum unique name. For example on Get-EventLog the shortest switch for -LogName is -LogN because if you tried -Log it would interfere with -Log which has a different function.

Get-EventLog -LogName application -After 01-06-2011 -Before 02-06-2011
Get-EventLog -LogName application -A “1 June 11″ -B “2 Jun 2011″
Get-EventLog -LogName application -Aft “1 - 06 - 11″ -Bef “2 / 6 / 2011″

Event Trace Logs (ETL)

Event Trace Logs (ETL) is EventLog version 2. There are a different set of cmdlet tools to hope with the new format of the logs and the additional data held in those logs. The new version is backward compatible with the old classic style of logging, so you could avoid using Get-EventLog in favour of Get-WinEvent. ETL was introduced in Vista and is available in operating systems after that. Load the Event Viewer using Start -> Control Panel -> Administration -> Windows Event Viewer. Under Windows Logs there are the classic logs, under that is the Application and Service Logs are more of the logs that are available with the Get-EventLog cmdlet, but are some how not classic.

We are going to be looking at the Trace and Debug logs. The first got-cha is that these logs don’t show up by default, you have to explicitly switch them on. From the Event Viewer panel highlight Application and Service Logs and file menu make sure that View -> Show Analytic and Debug Logs is checked. Application and Service looks like a place to contain logs from other companies. These new logs (or style of logs) split the data into different categories like Operation, Verbose, Analytic, etc. Operational logs seem to be switched on by default where as Trace, Diagnostic and others seem to be switched off by default. As you can see there are literally hundreds of log files available, so you can find information on almost any kind of issue that the system might be going through.

We are going to be looking at WMI activity so scroll down to WMI-Activity. When we click on the log icon (in the panel view) we get a list of the current log entries. Trace logs are switched off so we must enable it explicitly. Right-click on the log icon and select Enable Log. There is a scary warning about losing log entries when this is enabled so in Ed’s words: “it’s gonna whine, and I say yeh, yeh, yeh, so this is cool". When the scary message says “lose events” I think it means lose them from the view as opposed to them disappearing down a black hole. It would have been another: “Thanks Bill” for creating a logging system that is either disabled or almost-enabled! So let’s now list those logs via PowerShell.

The first thing to deal with is the got-cha I mentioned above. Just like the trace and diagnostic logs needed explicitly enabling in the Event Viewer; they need explicitly switching on in the PowerShell too. This is achieved with the -Force switch. The -Force switch works in combination with the -LogName switch. If you specify the -LogName name (in full) then Get-WinEvent will allow you access to that one log. If you use a wildcard in the -LogName only the Operational level log files will be used in the search space to find a match. If you use a wildcard in the -LogName and -Force then the log files, including all the Trace and Diagnostic logs, will be used in the search. The following 3 examples will help show the effects of this. Some of the entries may be blank because you don’t have the right privileges to see them. I have expanded the -LogName so that you can see the difference but when you do it they may be truncated, so don’t forget you can use Select-Object to only display the columns you want.

Get-WinEvent -ListLog “*CodeIntegrity*”

LogName                                 MaximumSizeInBytes    RecordCount  LogMode
-------                                 ------------------    -----------  -------
Microsoft-Windows-CodeIntegrity/Operational  1052672              4       Circular

Get-WinEvent -ListLog “*CodeIntegrity*” -Force

LogName                                 MaximumSizeInBytes    RecordCount  LogMode
-------                                 ------------------    -----------  -------
Microsoft-Windows-CodeIntegrity/Operational 1052672               4            Circular
Microsoft-Windows-CodeIntegrity/Verbose     1052672                            Retain

Get-WinEvent -ListLog “Microsoft-Windows-CodeIntegrity/Verbose”

LogName                                  RecordCount
-------                                  -----------
Microsoft-Windows-CodeIntegrity/Verbose  1052672

A bit of advice from Ed:

  1. “‘Use the Force Luke’, even if you’re name isn’t Luke you can still use the Force”
  2. “You’ll need to use the Force parameter only often enough, that when you do need to use it, you’ll probably forget about it, well at least that’s my behaviour anyway.”
  3. “You’ll have to actively train yourself to try and add this Force parameter, now if this is something you really don’t like, then you can actually write a function that wraps this cmdlet and automatically adds the Force to it, so that you don’t have to add it.”

The first bit of advice is obviously silly, bizarrely he kept saying “Use the force” like it was some kind of trigger but there’s no memory trigger for me that links make-my-command-work and Star Wars. The second bit of advice shows he is silly and the third bit of advice shows he gives silly advice.

The cmdlet Get-WinEvent has 2 roles: list the available logs (ListLog) and list the contents of a particular log (LogName). In almost the same way that you had to include -Force to make ListLog work you must use -Oldest to make -LogName work. Great, switches that don’t have to be specified but if you don’t use them, then it won’t work!

This particular log file is stored in chronological order which has oldest first. The default is newest first which is why it barfs when you don’t specify the -Oldest option to reverse the retrieval ordering. These log names increase the size of the command line so that it wraps. So now is probably a good time to see the PowerShell ISE - because it can wrap lines!

PowerShell ISE can be found in: Start Menu -> Programs -> Accessories -> Windows PowerShell ISE. After loading it in you will see 3 main areas: top, middle and bottom. The top section can hold a PowerShell script and run it line by line. The middle is all the output. The bottom section is almost the same as the regular PowerShell window in that you use the up and down arrows to go through your history. It looks like you practice writing PowerShell in the bottom panel, when you have got your command correct you can copy it to the script panel at the top. Do this for each line until you have a complete script which can be saved. Place the caret in the top panel at the line you would like to start running from and press the button with the green triangle (or F5) to start running. Each line will be echo’ed in the middle panel and if there are any errors, they will be displayed in red under the command. You can practice by entering one of the following lines into the bottom panel and hitting return. All the lines are equivalent. If we remember back to our magic parameters, in this case the magic parameter is Class i.e. if we add a command argument with no switch then this is the default switch to use.

Get-WmiObject -Class win32_bios
gwmi -Class win32_bios gwmi win32_bios

To give yourself something to look at in the PowerShell ISE enter the following command in to the top script panel and run it:

Get-WinEvent -LogName Microsoft-Windows-WMI-Activity/Trace -Oldest | where { $_.message -match ‘Class’ } | select message -ExpandProperty message

Don’t worry if the PowerShell ISE introduction didn’t make any sense, because it didn’t to me either! PowerShell ISE is better then the blue command box because it is more colourful and it allows you to save the command text separately. Other than that, PowerShell ISE is a pretty poor attempt at a integrated development environment. Ed Wilson is the Microsoft man for PowerShell and runs the HeyScriptingGuy blog site on Technet. He has a page devoted to logs and monitoring: http://blogs.technet.com/b/heyscriptingguy/archive/tags/event+logs/windows+powershell/logs+and+monitoring/

You can save an event log then you can use Get-WinEvent to parse that saved log. You can store all your event logs in a single location, use Get-ChildItem to do a directory listing of that location, open up all the files and look for a specific event. You might use this to correlate events for forensic analysis, where something has happened and you have to track it’s course through the system. Ed then takes us on a rough guide of his blog, and it looks pretty. He mentions a profiling cmdlet called Measure-Command that is used to measure the performance of different commands in order to optimise different solutions.

Questions and answers

Q: Are there tools to convert Korn shell scripts to PowerShell scripts?
A: No - not recommended because a Korn shell will be heavy on regular expressions for parsing text whereas PowerShell returns objects.

Q: Can I reboot a remote server with PowerShell?
A: Yes. There is a Stop-Computer and Restart-Computer cmdlet, so as long as you have connectivity and rights then you should be good. These commands take a list of computer names to use so you could reboot a list of computers.

Q: What is the difference between a single quote and a double quote?
A: A single quote(’) is a literal string and a double quote(") is an expanding string. The back-quote character (`) delimits the dollar and turns it into a normal character. Back-quote does the work of the back-slash in every other language I’ve come across.

$a=13
echo ‘a=$a’
a=$a
echo “a=$a”
a=69
echo “a=`$a”
a=$a
echo “number ‘$a’ is lucky for some”
number ‘13′ is lucky for some

Q: How can I sort a column after selecting?
A: Pipe it to the Sort-Object cmdlet specifying the column you wish to sort on. You can use the -Descending to change the order.

Get-Process | Sort-Object processname

Q: Can you cache the event log in memory before filtering, would that be faster?
A: You could read the entire event log into a variable and then you could do stuff. If you have lots of parsing and filtering then it could be faster. If you don’t have much to do then it’s much faster to do the filtering on the computer where the log sits.

Q: Can I do everything with events viewer using the PowerShell cmdlet?
A: Yes, if he was talking about doing everything in computer management then No.

Q: Can you use the up arrow to bring up a multi-line command statement in one?
A: No.

Q: Can you use PowerShell to monitor events as they occur?
A: Yes, there’s a whole series of articles on the HeyScriptingGuy site, mumble, mumble, mumble, look for the tag events, use can use WMI events or Windows events. [He just mumbled through the answer without saying anything coherent].

Q: Have you used Enter-PSSession and it’s related cmdlets?
A: Of course, I have. Search the HeyScriptingGuy site with the tag of Remoting.

Q: What is the difference between smooth brackets “()” and curly brackets “{}"?
A: Smooth brackets are used as parameters coming into methods opener and closers whereas curly brackets are typically used for script blocks or code blocks. I can use “@(” to create an array and “@{” to create a hashtable.

PowerShell array creation

$b = @("1″, “2″, “32″, “4″)

1
2
32
4

PowerShell hashtable creation

$m = @{"a” = “1″} + @{"b"="2″}

Name  Value
----  -----
a     1
b     2

PowerShell subexpressions

$x = 10
echo $($x * 10)
100