In this post, we will see how we can use inline transformers for writing flow variable content to file
Read More
Read More
Create Fixed width file with DataWeave in Mule 3.7
Read More
Read More
With Mule 3.7.0, a new modeling and transformation language was introduced, DataWeave. This is more powerful than its predecessor DataMapper which is marked as deprecated in Mule 3.7
DataWeave component appears as “Transform Message” in Mule Anypoint Studio under Mule Palette. DataWeave in Mule 3.7 supports processing of JSON, Java, CSV, EDI, XML, Pojo etc. One format that was missing in DataWeave support was Fixed Width text format. As per the beta release notes of Mule 3.8, DataWeave in Mule 3.8 is going to support fixed width text files. But for many of us, it may not be possible to upgrade Mule server for this.
So How to Process fixed width text file with DataWeave in Mule 3.7?
So for those who use Mule 3.7 with DataWeave and still want to process fixed width text files, here is the workaround for it.
Let’s consider below fixed with sample text file –
001MONSTARTWORK 001TUEPROGRESSWORK 001WEDTAKEABREAK 001THURESUMEWORK 001FRIFINISHWORK 001SATCHILLDOWN 001SUNGRABABEER
Record definition is –
Id - Start Position: 0, length: 3 Day - Start Position: 3, length: 3 Shout - Start Position:6, length: 13
Here is our sample flow that reads the file from input directory, applies dataweave transformation and writes the generated file to output directory.
Mule flow to convert fixed width text file to JSON
XML Flow –
<flow name="DW-FixedWidth-Processing">
<file:inbound-endpoint responseTimeout="10000" doc:name="File" moveToDirectory="output" path="input"/>
<dw:transform-message doc:name="Transform Message" metadata:id="44c29528-4f2b-4aa6-a454-bf2fe7b8de57">
<dw:input-payload doc:sample="string_1.dwl"/>
<dw:set-payload><![CDATA[%dw 1.0 %output application/json --- (payload splitBy "\n") map { Id:$[0..2], Day:$[3..5], Shout:trim $[6..18] } ]]></dw:set-payload>
</dw:transform-message>
<file:outbound-endpoint path="output" outputPattern="output.txt" responseTimeout="10000" doc:name="File"/>
</flow>
Let’s look at the DataWeave code now –
%dw 1.0
%output application/json
---
(payload splitBy "\n") map {
Id:$[0..2],
Day:$[3..5],
Shout:$[6..18]
}
As our payload is going to be file content i.e. String, we will split the content with new line char “\n”. Note the parenthesis used for `payload splitBy “\n”` which makes the output of that expression, input for map. Now, we have split our payload by newline so inside map we will be reading each line from our file and can be accessed with default alias $.
As per our record definition, Id field starts at 0 and is of 3 character long. So we map our Id by using Range Selector on String $[0..2] which is $[startIndex..endIndex]. Similarly we can select Day and Shout also.
We have intentionally used “\n” instead of “\r\n” to make sure our code works on windows as well as UNIX. Note that, for the last field our actual file content ends at 18th position but on windows formatted files, there should be “\r” at 19th position. As we will only map until last fixed position, “\r” should not be a problem. In case you see carriage returns in output then most probably you have included this last character in your output too.
Now, we have seen how we can split the fields in dataweave. You can actually write any logic on the fields. For example, to avoid any empty spaces in shout field we can use trim function.
shout: trim $[6:18]
Here is our output from above transformation –
[
{
"Id": "001",
"Day": "MON",
"Shout": "STARTWORK"
},
{
"Id": "001",
"Day": "TUE",
"Shout": "PROGRESSWORK"
},
{
"Id": "001",
"Day": "WED",
"Shout": "TAKEABREAK"
},
{
"Id": "001",
"Day": "THU",
"Shout": "RESUMEWORK"
},
{
"Id": "001",
"Day": "FRI",
"Shout": "FINISHWORK"
},
{
"Id": "001",
"Day": "SAT",
"Shout": "CHILLDOWN"
},
{
"Id": "001",
"Day": "SUN",
"Shout": "GRABABEER"
}
]
With this example, we learned how to consume fixed width text file with DataWeave in Mule 3.7.
Next see how to create fixed width flat file using DataWeave in Mule 3.7
Feel free to let me know what you think or any questions, I will be happy to help.
Apr
12
2016
How to deploy wordpress on tomcat
Read More
Read More
How to configure Mule SMTP connector using iCloud on Mac?
Read More
Read More
With Spring 3.2, MVC testing is now integrated in the framework itself as Spring-test project. These are simple and easy ways to test your MVC. A simple introduction can be read here.
A very common scenario in any web application is dependency on User Authenticated Sessions in-between multiple requests.
For a usual MVC testing we can write a test case to test functionality. We will see here how we can use applications Authentication service to authenticate user and use its session across multiple requests.
Here is a pom.xml –
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <artifactId>SpringMVC</artifactId> <packaging>war</packaging> <name>SpringMVC</name> <dependencies> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-webmvc</artifactId> <version>3.2.0.RELEASE</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-test</artifactId> <version>3.2.0.RELEASE</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> </dependency> <dependency> <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> <version>3.0-alpha-1</version> <scope>provided</scope> </dependency> <dependency> <groupId>com.jayway.jsonpath</groupId> <artifactId>json-path</artifactId> <version>0.8.1</version> <scope>compile</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-mapper-asl</artifactId> <version>1.9.11</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>3.2.0.RELEASE</version> </dependency> <dependency> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-core-asl</artifactId> <version>1.9.11</version> </dependency> </dependencies> <version>1.0.0</version> <groupId>com.mms.blogs</groupId> </project>
Let’s configure Spring MVC. Here is SpringMVC-context.xml –
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd"> <mvc:annotation-driven/> <context:component-scan base-package="com.mms.blogs.demo.SpringMVC"/> <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="prefix"> <value>/WEB-INF/jsp/</value> </property> <property name="suffix"> <value>.jsp</value> </property> </bean> </beans>
web.ml
<?xml version="1.0" encoding="UTF-8"?> <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" id="WebApp_ID" version="3.0"> <display-name>SpringMVC</display-name> <context-param> <param-name>contextConfigLocation</param-name> <param-value>/WEB-INF/classes/SpringMVC-context.xml</param-value> </context-param> <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener> <servlet> <servlet-name>SpringMVC</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>SpringMVC</servlet-name> <url-pattern>/*</url-pattern> </servlet-mapping> <welcome-file-list> <welcome-file>index.html</welcome-file> <welcome-file>index.htm</welcome-file> <welcome-file>index.jsp</welcome-file> <welcome-file>default.html</welcome-file> <welcome-file>default.htm</welcome-file> <welcome-file>default.jsp</welcome-file> </welcome-file-list> </web-app>
Until here was all configuration part of the application. Now let’s see the java classes involved in our testing.
User, a value object that will hold the user information and will reside in session object once user passes authentication.
package com.mms.blogs.demo.SpringMVC.vo; public class User { private String userid; private String firstName; private String lastName; public String getUserid() { return userid; } public void setUserid(String userid) { this.userid = userid; } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } }
Let’s take a look at LoginController. This will be an entry point to our application where User will be authenticated and a session object will be set for the user. Subsequent requests from this user will expect a User object in session with his information. More info on Session Attributes can be read on http://vard-lokkur.blogspot.com/2011/01/spring-mvc-session-attributes-handling.html
Note a declaration of @SessionAttribute with key as “user” and type as “User”. This means, when we create a User object and put it in return model with key “user”, framework will synchronize this object with Session and put it in session.
package com.mms.blogs.demo.SpringMVC.controller; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.SessionAttributes; import com.mms.blogs.demo.SpringMVC.vo.User; @Controller @RequestMapping(value="/login",method=RequestMethod.POST) @SessionAttributes(value="user",types=User.class) public class LoginController { @RequestMapping(value="/authenticate",method=RequestMethod.POST) public String authenticate(@RequestParam String userid, Model model){ // effectively you can put your any LoginService here to authenticate user User user = new User(); user.setUserid(userid); user.setFirstName("Manik"); user.setLastName("M"); // Since we have defined SessionAttribute user, this object will be stored at SessionLevel model.addAttribute("user", user); return "Home"; } }
Now as we have LoginController ready, lets see another controller that will consume this User session to print user name. Here is a simple EchoUserController that gets the User object from session and returns to caller. something to note here –
- This declares a SessionAttribute user, so will expect one pre-set in session.
- getUser method has a argument with @ModelAttribute(“user”). This means when this method is called, framework will expect a session attribute with key “user” and type “User”. If this exists, framework will pass it to the method else it will throw an exception for missing session attribute and call to invoke this method will fail.
- For some reason, if session attribute with key “user” exists but value is null then we can implement our own check.
package com.mms.blogs.demo.SpringMVC.controller; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.validation.BindingResult; import org.springframework.web.bind.annotation.ModelAttribute; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.ResponseBody; import org.springframework.web.bind.annotation.SessionAttributes; import com.mms.blogs.demo.SpringMVC.vo.User; @Controller @SessionAttributes(value="user",types=User.class) @RequestMapping(value="/echo") public class EchoUserController { @RequestMapping(value="/getuser",method=RequestMethod.POST) public @ResponseBody User getUser(@ModelAttribute("user") User user, BindingResult result, Model model){ if(user == null){ throw new RuntimeException("User not found"); } return user; } }
So we are all set with a layout to test our requests using Spring MVC test. Look at the below test class and we will go over each test case –
package com.mms.blogs.demo.SpringMVC.Controller; import org.junit.Before; import org.junit.Test; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.mock.web.MockHttpSession; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; import org.springframework.test.context.web.WebAppConfiguration; import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MvcResult; import org.springframework.test.web.servlet.RequestBuilder; import org.springframework.test.web.servlet.ResultActions; import org.springframework.test.web.servlet.request.MockMvcRequestBuilders; import org.springframework.test.web.servlet.result.MockMvcResultHandlers; import org.springframework.test.web.servlet.result.MockMvcResultMatchers; import org.springframework.test.web.servlet.setup.MockMvcBuilders; import org.springframework.web.context.WebApplicationContext; @WebAppConfiguration(value="src/main/webapp") @ContextConfiguration(locations={"classpath:SpringMVC-context.xml"}) public class EchoUserControllerTest extends AbstractJUnit4SpringContextTests { @Autowired private WebApplicationContext webapp; private MockMvc mockMvc; @Before public void setup(){ this.mockMvc = MockMvcBuilders.webAppContextSetup(webapp).build(); } @Test public void testAuthenticate() throws Exception{ this.mockMvc.perform(MockMvcRequestBuilders.post("/login/authenticate") .param("userid", "MMS1")) .andExpect(MockMvcResultMatchers.status().isOk()) .andExpect(MockMvcResultMatchers.model().attributeExists("user")) .andDo(MockMvcResultHandlers.print()); } @Test public void testGetUser1() throws Exception{ this.mockMvc.perform(MockMvcRequestBuilders.post("/echo/getuser")). andExpect(MockMvcResultMatchers.jsonPath("$.userid").value("mms")); } @Test public void testGetUser2() throws Exception{ ResultActions auth =this.mockMvc.perform(MockMvcRequestBuilders.post("/login/authenticate") .param("userid", "MMS1")); MvcResult result = auth.andReturn(); MockHttpSession session = (MockHttpSession)result.getRequest().getSession(); RequestBuilder echoUserReq = MockMvcRequestBuilders.post("/echo/getuser").session(session); this.mockMvc.perform(echoUserReq) .andDo(MockMvcResultHandlers.print()). andExpect(MockMvcResultMatchers.jsonPath("$.userid").value("MMS1")); } }
setup() method will initialize and build a MockMVC object for us using our webapp and configuration.
testAuthenticate() tests our LoginController.authenticate(). It passes userid to authenticate and setup session. This test should pass.
testGetUser1() tests our EchoUserController.getUser() method. This test case call getUser and expects value of userid property in returned User object to match with “MMS1”. So as we have seen earlier, getUser method expects a session attribute user. Every request in MockMVC is independent of other. Calling testAuthenticate() first and the testGetUser1() will run different contexts and hence session set by former will not be visible to later. Here, testGetUser1() is expected to fail for missing user attribute in session.
testGetUser2() agains tests getUser() method but here difference is we will be providing authenticated session to it. To do so, we will call authenticate method first –
ResultActions auth =this.mockMvc .perform(MockMvcRequestBuilders.post("/login/authenticate") .param("userid", "MMS1"));
This call will authenticate user and set up a session for it. Let’s get the result of it in ‘auth’ of type ResultAction.
Second step is now to retrieve Session object from this response to use further.
MvcResult result = auth.andReturn();
MockHttpSession session = (MockHttpSession)result.getRequest().getSession();
So we got an authenticated session for this user. Now we can pass this session to any other request. Let’s call our getUser() method again but this time setting session on the request.
RequestBuilder echoUserReq = MockMvcRequestBuilders.post("/echo/getuser").session(session);
Here, we build our request and attach a session to it. With this we have made an authenticated session available to this request to getuser. Once done, below test step is expected to pass, returning User object.
this.mockMvc.perform(echoUserReq) .andDo(MockMvcResultHandlers.print()). andExpect(MockMvcResultMatchers.jsonPath("$.userid").value("MMS1"));
We are done! See, it’s so easy to use sessions in testing. This also makes sure we are testing our getUser with real sessions created by actual LoginController and not any dummy session created for testing purpose where there is a chance that it may differ from actual sessions created by application via LoginController.
Enjoy!!
Older post are available in the archive
