The methodology chapter
It is important to have a clear understanding of the purpose of each chapter of the project report. From one point of view, the answer is always the same: you are clearly communicating the content of the project to the Reader. Remember that your task as the author of the report is to make it as easy as possible for the Reader to understand the project. You can assume that the Reader is intelligent, educated and familiar with theoretical concepts; however, you cannot assume that the Reader has a deep knowledge of the concepts or techniques that you have used in your project. You need to equip the Reader with that detailed understanding - or at least direct them to more detailed coverage by providing references - to be sure that they will be able to follow the discussion of your results.
In most cases, the report will contain an explicit methodology chapter which sets out the methods used in the project. It will discuss any options that were considered, and will present a rationale for the selected option. It will also explain how a particular method is applied. It is usually not sufficient just to name a method; it needs to be clear to the Reader that you fully understand the method and how to apply it in practice.
The discussion of methods can get quite detailed, and to help the Reader navigate the content, a top-down approach is recommended starting with an overview as described below. In general, there are four types of method that need to be covered in your methodology chapter. They are the methods for
- The literature review
- Domain-specific activities
- Project management
If you start by carefully considering what the Reader needs to know at each stage of the report, you will be able to make some good decisions about structure, content, level or detail and so on. When arriving at the start of the methodology chapter, one of the first things the Reader needs is a reminder of the aim of the project and how the literature review (which they have just read) helps to inform the practical work.
This is also a good time to set some general parameters for the project by explicitly defining the project approach. You might take an experimental approach, for example, by collecting data under different conditions to see the effect on the results. Alternatively, you might take an action research approach in which you carry out some carefully-controlled action in a real environment to see whether it produces the hypothesised effects. By defining your approach, you set some expectations for the structure of the methodology and for the type of results that the Reader will be presented with later on. Various approaches are described in a little more detail in some of the other pages in this section of the notes.
Next, the Reader need some orientation - a broad-brush picture of the stucture of the practical work. Often, the practical work progresses through a series of steps, and for that reason it is often a good idea to provide a flow diagram that shows the relationships between those steps. The steps themselves are often defined by the objectives that you set earlier. It helps the Reader to keep a clear perspective on what you are doing if you can exploit structures that you have introduced at an earlier stage. Having a clear correspondence between the major steps in the project process and the objectives is therefore a good idea. Presenting the structure as a flow chart at this stage, for example, can help to structure the remainder of the chapter. You can exapnd each main element in the diagram in its own subsection.
Literature review methods
Often overlooked, it is good to tell the Reader how you found an selected the references that you used in the literature review. This section does not need to be very long but it reassures the Reader that the literature review was thorough, and that you applied good judgement in your selections. You could cover, for example
- which databases or repositories you used (e.g. Google Scholar, ACM, IEEE, etc)
- which search keywords and filters you used
- the total number of references that were found
- how you filtered the total number down and how you selected the references that you eventually used
This is the section that demonstrates your command over your subject area. If you are developing software, for example, this section would cover your selection of technologies and tools, your use of practices such as version control and unit testing and any principles that you rely on in your software design and implementation. This might include references to software engineering principles such as SOLID, or general approaches such as clean code. On the other hand, if your project is about human factors in computing, you might be conducting interviews with professionals in different computing roles. The methods you need to cover in that case are those related to the preparation of interview questions, managing the interview process and coding and analysing transcripts.
From an academic point of view, this is the most important section of your methodology since it is through the evaluation process that you directly address the aim of the project. For many computing students, however, evaluation can seem like an annoying extra hoop to jump through. Typically, students are more focussed on the practical activities of building software, designing interfaces, carrying out security tests, etc. It is important to think about those activities as a means to an end rather than the end in itself. As noted above, they should be done well, and the methods you use need to be explained, but they only generate the data for you to analyse.
Take, for example, a simple case of a project in which the aim is to test the effectiveness of a particular user interface design. The UI needs to be designed using recognised techniques - that way we know that the student has done a thorough job and there is indeed something substantial to evaluate; the UI has not just been hacked together with no control. To test its effectiveness, though, the student needs to apply some new techniques to determine the effectiveness of the UI that has been produced. A scale of effectiveness needs to be established, data needs to be collected about the UI in use, and that data has to be processed to place the UI somewhere on the established scale. This all needs to be done in a controlled way so that the final conclusions are credible.
Evaluation can be quantitative whereby the student collects numerical data and performs some kind of mathematical analysis to arrive at a final result. It can also be qualitative in which data is collected from people in the form of survey responses or interview transcripts, for example. Knowing what type of data you are dealing with determines the types of analysis that you can do. This is discussed in more detail in the data section.
When discussing how the evaluation will be done, it should be clear how the selected methods relate to the project aim and to any research questions. It can also be helpful to suggest to the Reader what form the final results will take and how they will eventually be interpreted. This is not to pre-emp the findings; it is just part of a full explanation of the process you will go through.
Project management methods
Academic projects come in different sizes ranging from a pratical coursework exercise at undergraduate level, through Honours projects and Masters dissertations up to PhD projcts which span several years. In each case, part of the purpose of the project is to demonstrate that the student can manage an activity of the relevant complexity. This is done through the application of appropriate project management techniques.
Just like the other types of method, there are some project management techniques that are appropriate for your project, and some that are not. For example, many software engineering students suggest using an Agile methodology such as Scrum in their Honours or Masters projects. This is a natural inclination: Scrum is one of the best-developed and most widely-used forms of Agile software development. However, many aspects of Scrum are designed to facilitate collaborative work in small teams. That is not the context in which the student is working. The Honours project and the Masters dissertation are both individual projects, and none of the teamworking aspects of Scrum are relevant. Claiming to be using Scrum with no further qualification implies one of two things: either the student does not understand the purpose of Scrum, or they have not really put much thought into selecting appropriate methods for the project. Either way, at this point the Reader starts to doubt the quality of the project.
Some aspects of a formal methodology such as Scrum are relevant in an individual project. The process of building the software could be broken down into sprints, for example, or the work of the project could be represented as a prioritised task backlog. It is important, though, to be precise about which techniques are being used and for what purpose.
Explaining you methods is difficult. However, it is an essential part of any academic project if you want the Reader to have faith in your final conclusions. As well as setting out your methods, you also need to demonstrate that they were followed correctly by providing evidence, usually as appendix material.
The four types of method that should be covered are:
- Literature review methods
- Domain-specific methods
- Evaluation methods
- Project management methods
Focusing explicitly on each type can help to structure a methodology chapter