Spring Security OAuth2 Resource Server Always Returning Invalid Token

I am trying to get a basic in-memory OAuth2 server running using the Spring Libraries. I have been following the sparklr example.

I currently have configured the Server and almost everything is working, however I cannot access my restricted resource from the resource server.

My test workflow:

1. 
   

   Access the oauth authorized URI to start the OAuth2 flow: http://localhost:8080/server/oauth/authorize?response_type=code&client_id=client

   
   


2. 
   

   Redirect to the login page: http://localhost:8080/server/login

   
   


3. 
   

   Handle the approval and redirect to my configured redirect page w/ a code parameter: http://localhost:8080/client?code=HMJO4K

   
   


4. 
   

   Construct a GET request using Basic Auth using the client id and secret along with the grant type and code: http://localhost:8080/server/oauth/token?grant_type=authorization_code&code=HMJO4K

   
   


5. 
   

   Receive an access_token and refresh token object in return

   
   

   { access_token: "f853bcc5-7801-42d3-9cb8-303fc67b0453" token_type: "bearer" refresh_token: "57100377-dea9-4df0-adab-62e33f2a1b49" expires_in: 299 scope: "read write" }

   
   


6. 
   

   Attempt to access a restricted resource using the access_token: http://localhost:8080/server/me?access_token=f853bcc5-7801-42d3-9cb8-303fc67b0453

   
   


7. 
   

   Receive an invalid token reply

   
   

   { error: "invalid_token" error_description: "Invalid access token: f853bcc5-7801-42d3-9cb8-303fc67b0453" }

   
   


8. 
   

   POST to the token uri again to refresh token: http://localhost:8080/server/oauth/token?grant_type=refresh_token&refresh_token=57100377-dea9-4df0-adab-62e33f2a1b49

   
   


9. 
   

   Receive a new token

   
   

   { access_token: "ed104994-899c-4cd9-8860-43d5689a9420" token_type: "bearer" refresh_token: "57100377-dea9-4df0-adab-62e33f2a1b49" expires_in: 300 scope: "read write" }

   
   

I am really not sure what I am doing wrong, but it appears that everything other than accessing the restricted uri is working. Here is my configuration:

@Configuration
public class Oauth2ServerConfiguration {

    private static final String SERVER_RESOURCE_ID = "oauth2-server";

    @Configuration
    @EnableResourceServer
    protected static class ResourceServerConfiguration extends ResourceServerConfigurerAdapter {

        @Override
        public void configure(ResourceServerSecurityConfigurer resources) {
            resources.resourceId(SERVER_RESOURCE_ID);
        }

        @Override
        public void configure(HttpSecurity http) throws Exception {
            http
                .sessionManagement()
                    .sessionCreationPolicy(SessionCreationPolicy.IF_REQUIRED)
                .and().requestMatchers()
                    .antMatchers("/me")
                .and().authorizeRequests()
                    .antMatchers("/me").access("#oauth2.clientHasRole('ROLE_CLIENT')")
            ;
        }
    }

    @Configuration
    @EnableAuthorizationServer
    protected static class AuthotizationServerConfiguration extends AuthorizationServerConfigurerAdapter {

        @Autowired
        private ClientDetailsService clientDetailsService;

        @Autowired
        @Qualifier("authenticationManagerBean")
        private AuthenticationManager authenticationManager;

        @Override
        public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
            clients.inMemory()
                .withClient("client")
                    .resourceIds(SERVER_RESOURCE_ID)
                    .secret("secret")
                    .authorizedGrantTypes("authorization_code", "refresh_token")
                    .authorities("ROLE_CLIENT")
                    .scopes("read","write")
                    .redirectUris("http://localhost:8080/client")
                    .accessTokenValiditySeconds(300)
                    .autoApprove(true)
            ;
        }

        @Bean
        public TokenStore tokenStore() {
            return new InMemoryTokenStore();
        }

        @Override
        public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
            endpoints
                .tokenStore(tokenStore())
                .userApprovalHandler(userApprovalHandler())
                .authenticationManager(authenticationManager)
            ;
        }

        @Override
        public void configure(AuthorizationServerSecurityConfigurer oauthServer) throws Exception {
            oauthServer.realm("oauth");
        }

        @Bean
        public ApprovalStore approvalStore() throws Exception {
            TokenApprovalStore store = new TokenApprovalStore();
            store.setTokenStore(tokenStore());
            return store;
        }

        @Bean
        public UserApprovalHandler userApprovalHandler() throws Exception {
            TokenStoreUserApprovalHandler handler = new TokenStoreUserApprovalHandler();
            handler.setRequestFactory(new DefaultOAuth2RequestFactory(clientDetailsService));
            handler.setClientDetailsService(clientDetailsService);
            handler.setTokenStore(tokenStore());

            return handler;
        }
    }
}

Is there something I am missing or am I approaching this incorrectly? Any help would be greatly appreciated.