Building Web Applications with Flight: Part 3

This is Part 3 of a series in which we are looking at how to design and build a complex, data driven application using Flight, the JavaScript framework that Twitter.com runs on. The aim is to produce an analytics package which will show emerging micro-trends within Twitter search results.

In Part 2, we created some user stories, event flows and set out the requirements for two components to satisfy one of our stories.

In Part 3 we will add a keyword input form to our index page, allowing the user to set the keywords for the search.

This requires creating two components: a UI component to handle the submit event and a data component to process the input. In the process, we will become more familiar with creating components, see how to listen for and trigger events and how to pass data with those events.

Required knowledge

This article assumes that you have a working knowledge of JavaScript and jQuery, as well as Flight and Jasmine. If you’re new to Flight, check out the Flight documentation or grab a copy of my book, Getting Started with Twitter Flight. You might also want to read up on Jasmine, Jasmine-jQuery and Jasmine-Flight.

Creating a UI component

In Part 2, we defined the HTML for our keyword search form. You’ll need to copy that and paste it in to index.html, replacing the H1 tag you added in Part 1. The complete file should look like this:

Next, we need to create a component that will listen to submit events from the form, prevent the default behaviour, get the new keywords and trigger the ui-keywords event. We’ll call this component search-form.

To create a scaffold for the component, use the flight generator, like in the hello-world example in Part 1:
$ yo flight:component search-form

This will create two files: /app/js/component/search-form.js and /test/spec/component/search-form.spec.js. Open search-form.spec.js in your editor.

Testing

When testing components it is important to test the component’s interface and not its internal workings. This allows you a free hand when you come to writing the code and ensures that you don’t need to modify the tests if you refactor a component’s inner workings.

“Test the interface, not the implementation”

From an external point of view, this component only does one thing: it listens for submit events and, as a response, triggers ui-keywords events. This is its interface.

Thus our test looks like this:

We could do some validation of the input at this stage; e.g. checking for empty strings. For now (to keep this simple) we’ll assume the user is not going to make any mistakes.

Use karma to run the tests, as in Part 1:
$ npm run watch-test

You should see them failing with the following error:
Screen Shot 2014-02-12 at 1.23.48 PM

This error occurs because we are not preventing the bubbling of the submit event to the browser, so it thinks we’re actually submitting the form and reloads the page.

To fix this we will need to the search-form component to handle the event and prevent the default behaviour. This is pretty straightforward, so I’ll just give you the code up-front.

If this (or anything else in this series) requires clarification, feel free to contact me on Twitter:


Once you are done, take a look at your terminal. You should see that the tests are passing, which means our component is complete. That means it is time include this fully armed and operational component in our app.

Open up /app/js/page/default.js, import the search-form component and attach it to .js-search-form. You can strip out the hello-world stuff while you’re there. default.js should now look like this:

Time to check it out in a browser.
$ open http://localhost:8080
If your server isn’t running, start it like so:
$ npm run server

Open DevTools and make sure the cache is turned off by opening the DevTools options, selecting options (top right) and checking the “Disable cache (while DevTools is open)”.

Now, reload the page, type something in the box and submit the form. You should see something like this:

Screen Shot 2014-02-22 at 10.21.31 AM
The event logs show

  • we are listening for a submit event
  • on the form.js-search element
  • from a component named searchForm

and

  • A ui-keywords event has been triggered
  • with data {keywords: 'test'}
  • on the form.js-search element
  • from a component named searchForm

Event logs are incredibly useful for tracking down issues in an application, or when trying to understand how an app fits together. You can immediately see which component triggered an event, and the element it is attached to.

Data

The UI side of this story is now complete, so we can move on to the data processing. In Part 2, we specified that the input is a space-separated string of keywords. We need to listen for the ui-keywords event, split the keywords string in to an array of keywords and then rebroadcast it as a data-keywords event.

We’ve already been through the basics of how to build components twice, so I’m just going to give you a hint ($ yo flight:component), suggest a name for the component (keyword-processor) and a test to implement (“Should listen for ui-keywords and trigger data-keywords”).

And, don’t forget, you’ll need to attach the component to the DOM in default.js

Here’s the completed code for you to check yours against.

Spec

Component definition

Page

Coming soon: using templates

In Part 4, we’re going to leave the search & analytics and instead create a UI component that will use a template system to render some dummy results.

Building Web Applications with Flight: Part 2

This is Part 2 of a series in which we are looking at how to design and build a complex, data driven application using Flight, the JavaScript framework that Twitter.com runs on. The aim is to produce an analytics package which will show emerging micro-trends within Twitter search results.

In Part 1 of this series, we set up an environment and generated a Flight application using the Yeoman Flight Generator. We used a simple http server to launch the app and used a Flight component to change the title to “Hello World”.

In Part 2, we’re going to take a step back from the code and look at application design. We will create some user stories, sketch out event flows and set out the requirements for some components.

Behaviour Driven Development

Flight is uniquely well suited to behaviour-driven development because it is possible to exhaustively test a component’s behaviour. A good way to decide which behaviours we want a component to possess is to start with a user story, model the event flow and, finally, decide how to componentize that flow.

User stories

I’m not going to go to great lengths with the user stories as they are quite straightforward. If they or anything else in this series fails to make sense, don’t hesitate to get in touch via Twitter:

Let’s start with our basic premise:

I want to see emerging micro-trends within Twitter search results.

This is a pretty epic story and it’s also pretty vague, so lets break that down in to some more concrete features.

I want to perform a keyword search

I want to see a list of trends that occur within the search results

I want the trends to be updated over time

For this application, we can define a trend simply as an entity (keyword, user, url, photo) that appears often in the search results.

Let’s take the first story, performing a keyword search, and break it down a little further. First of all we’ll need some keywords to search with:

I need to set keywords to I can perform a keyword search

And then we’re going to need authorization to perform a search through Twitter’s API. This could be done as a simple application auth, which would allow access to a restricted subset of API methods. However, we’re going to ask the user to authorize the application in a 3-legged OAuth process. This will give us full access, including access to the streaming API.

I need to authorize the app via Twitter so I can stream search results

Note that we’re not expecting to see the results of the search in the UI. Our app will show trends, not the tweets themselves.

So, which story shall we take first? Well, OAuth is pretty complex and we really just want to get started building something, so let’s take the keywords story as a starter. It looks pretty straightforward.

Setting Keywords

What is the absolute minimum a user needs to be able to set keywords? A form with a text input field and a submit button.

When the user submits the form, we’ll need to interrogate the text field to to get the keyword string. Once we have the keyword string, we will break it in to keywords. Note that there’s no requirement at this stage to actually perform the search.

Defining an event flow

Events need to describe interesting moments in an application process, while trying to keep a separation between data (processing, ajax) and ui (clicking submit, displaying content). 

The way I see it, the following is going to happen:

  1. User clicks submit
  2. Keyword data is gathered
  3. Keyword data is processed

That breaks down in to three interesting moments (events):

  1. User has clicked submit
  2. keywords have been gathered
  3. keyword input has been processed

The submit event comes from the browser and so will be handled by a UI component. That component then needs to gather the keywords as a string and trigger a UI event, ui-keywords. Then the keyword string data needs to be processed to reveal individual keywords. This will be handled by a data component which, when it has finished, will trigger a data event containing individual keywords, data-keywords.

Decoupling UI & Data

You might wonder why we need to separate the processing of the input from the handling of the submit event, rather than doing it all in a single component. This decoupling might be useful to us in the future if we want to process keywords from another source such as saved searches. It also makes the components smaller and thus easier to design and to test.

Of course, there is a trade-off to be had between lots of small components (so many components!) and fewer, larger ones (what does this do?). I prefer smaller components. If you do get in to trouble, it’s easier to merge components than break them apart.

Define Event Requirements

There’s just one more thing to do before we start coding. We need to define the format for each of our events. Here this is done for clarity but if you were working in a team, doing this in advance would allow each developer to work on related components simultaneously.

The submit event is already defined by the browser, but we do have to define the form HTML so we know how to find the input element. Here’s some simple HTML for the form:

<form class=”js-search”>
<input class=”js-keywords” type=”text” />
<input class=”js-submit” type=”submit” />
</form>

And here are our events:

ui-keywords, {
keywords: String
}

data-keywords, {
keywords: Array[String]
}

Part 3

In Part 3, we’ll build two components, a UI component to handle the submit event and a data component to process the input.

Building Web Applications with Flight: Part 1

This is the first post in a series which provides an in-depth look at how to build complex applications using Flight, a lightweight, scalable web application framework from Twitter.

Flight

If Flight is new to you, check out this presentation: Dan Webb on Flight at JSConf 2013 It gives a good overview of what it is and why it is awesome. Then take a look at twitter.github.io/flight and the documentation at github.com/flightjs/flight. If you want a more in-depth look at Flight, try my book, Getting Started with Twitter Flight.

Flight has a growing developer community and a range of plugins to extend the core functionality. Most of these are available with Bower. A list can be found at flight-components.jit.su.

About This Series

In this series we will use a generator to create a scaffold and then use test-driven development to flesh out a full-blown flight application. We will look in detail at creating pages, components & mixins, designing event flows, writing tests with Jasmine, rendering HTML using templates (both client-side and server-side) and managing UI interactions, API calls and data. We’ll use NodeJS to create a web server so we can access the Twitter API with OAuth and use Socket.io to manage a websocket to stream Tweets to the client.

But, before all that…

Setting up your environment

Flight doesn’t require any special environment. You can download the standalone Flight package and simply insert it in to your application as you would any JavaScript library. However, in this series, we’ll be looking at how to build a full, flight-based application. For that we’re going to need some package management tools and, eventually, an application server.

We’re going to be using the Node Package Manager (npm) and Bower to manage development and application packages respectively, and will eventually use NodeJS to create an application server to manage templates & Ajax requests

  1. Installing Node
    To get started, we’ll need to install NodeJS. I recommend doing this through the Node Version Manager (NVM). This will allow you to maintain multiple versions of Node simultaneously. On the command line:

    $ curl https://raw.github.com/creationix/nvm/master/install.sh | sh
    $ nvm install 0.10

    This will install NVM and the latest v0.10.x release of Node.
  2. Install the Flight generator
    The Flight generator uses Yo, part of the Yeoman workflow.

    $ npm install -g yo
    $ npm install -g generator-flight

You’ll also need a good text editor. SublimeText is very good, though for complex JavaScript like this you may want an IDE with better code completion and validation. My preference is JetBrains WebStorm.

About the Application

You’ve probably had it up to here with todo apps, so why don’t we take on something a little more adventurous? We’re going to build a real-time analytics app for Twitter which will allow you to gather interesting data from search results via the Twitter streaming API. I hope this will provide insight in to how to structure a complex application and how to manage data-driven applications with Flight.

The purpose of the application will be to surface micro-trends within a specific search. Say there’s some big event such as an earthquake or the Superbowl. There tend to be huge numbers of Tweets generated about this sort of event. So many in fact that it is nigh on impossible to find the ones that matter. Our app will analyze all these Tweets and try to identify recurring phrases, hashtags, users, links and retweets within the search results. Over the course of a few minutes, trends will start to emerge. Those trends will then evolve over time, allowing you to identify emerging stories and stay ahead of the curve.

Creating a scaffold

We can start by using generator-flight to create a scaffold and get a really simple “Hello, world!” up and running. If you’re experienced with Flight, jasmine-flight and generator-flight, you can skip this and proceed straight to lesson 2, but it’s probably still worth having a glance through the code examples below to make sure we’re all on the same page.

Create a folder for your application and use the generator to scaffold it out:

$ mkdir twitter-analytics && cd $_
$ yo flight twitter-analytics

The generator will ask whether you want normalize, bootstrap and google analytics. Just say no to all of that. We’ll worry about those later.

To check everything’s working as expected, start up the static file server that the generator has configured for you:

$ npm run server

Then, in a new terminal

$ open http://localhost:8080

Hopefully you’re using Chrome and if not, I’d really recommend it; the web developer tools are second to none. For the sake of these articles, I’ll assume you are. If you open the javascript console in Chrome, you should see this:

Screen Shot 2013-12-23 at 3.25.49 PM

Doing it with components

Currently the “Hello world” message is hard-coded in /app/index.html. We’re going to create a component that inserts that text dynamically on page load. This isn’t a real-world scenario but will provide a glimpse in to components; how they are initialized, how they can modify the DOM and how to test them.

Create a component definition & test spec

Back on the command line, in the root of your application, enter the following:
$ yo flight:component hello-world
This creates two files: the component definition/app/js/components/hello-world.js and the component spec/test/spec/component/hello-world.spec.js.

The spec

The spec is a Jasmine unit test file, utilizing the jasmine-jquery and jasmine-flight extensions.

Test-driven development and Flight

I love test-driven development. A well written test suite describes why a module exists and what it is supposed to do, making life easier for everyone involved.

Flight components are particularly well suited to behaviour-driven development because they interact with the world in very specific ways: DOM interrogation & manipulation, DOM events and external API calls. Each of these can be tested exhaustively and easily. Pretty much every test you create will follow one of these patterns:

  • When I trigger an event on a node, an event is fired
  • When I trigger an event on a node, the DOM is altered
  • When I trigger an event on a node, an API is utilized
  • When I instantiate a component, an event is fired
  • When I instantiate a component, the DOM is altered
  • When I instantiate a component, an API is utilized
  • When a response from an API is received, an event is fired

Jasmine, jasmine-jquery and jasmine-flight make this easy by providing methods to instantiate components, and assertions which listen for events, interrogate the DOM, handle asynchronous functionality and ajax requests & responses.

What not to test

I think it is also important to understand what not to test. All the scenarios above test what effect the component has on the DOM or on an API. It should never be necessary to test the internal workings of a component; whether a specific method is called, or if a component has a particular internal state. These things are irrelevant to the effect the component has on the application. Behaviour-driven development is a good way to avoid such irrelevancies, as tests are only able to specify how the component should work at a high level.

Reading the spec

Thanks to Jasmine’s simple syntax the spec should be easy to read but it’s worth going over the details if you’re unfamiliar with either Jasmine or jasmine-flight. Let’s look at the spec line by line.

describeComponent('component/hello-world', function () {

describeComponent is a method provided by jasmine-flight. The first parameter is a path to the component we’re going to test. describeComponent will load the component and provide us with some utility methods to manage it within our tests.

beforeEach(function () {
setupComponent();
});

In beforeEach (which will be executed before each test), we call one of jasmine-flight’s utility methods, setupComponent. This creates an empty DIV within the body of the page and attaches a new instance of the component to it. It also provides a reference to the component instance which is available within the tests as this.component.

describeComponent also implements a teardown method which will destroy the component and remove the DIV after a test is run, ensuring the next test has a clean slate to work on.

Now on to the tests themselves.

it('should be defined', function () {
expect(this.component).toBeDefined();
});

This test says it expects this.component to be defined. This test should pass. A failure here indicates that the path provided to describeComponent does not reference a valid Flight component.

Running the test suite

generator-flight sets up a test runner for you using Karma. You can run the tests from the command line. At the root of your project:
$ npm run watch-test

There is currently an issue in generator-flight which means karma may have the wrong path for jquery, resulting in this warning:

WARN [watcher]: Pattern “/Users/thamshere/open-source/test/app/bower_components/jquery/jquery.js” does not match any file.

If you see this warning, amend line 22 of karma.conf.js to read
'app/bower_components/jquery/dist/jquery.js',

Thanks to @franzheidl for the heads up.

This will fire up an instance of Chrome and run the test suite. In the command line output you should see “Executed 2 of 2 SUCCESS”. Which is good.

Karma runs continuously in the background and will re-execute the tests whenever you make a change in a javascript file in the application. To see this in action, try changing the second test so that it fails:

Back on the command line, you should see that the tests have been re-executed and failed:

Screen Shot 2014-02-13 at 10.58.22 AM

Writing a test

First of all, we want to figure out how our hello-world component should behave. In this case, we’re going to say there should be an <h1> element on the page and our component should insert the text “Hello, world!” in to it. So our test needs to check that somewhere on the page, an h1 element contains that text. Replace the failing assertion above with the following:

it('should set content of the h1 tag to "Hello, world!", function () {
expect($('h1')).toHaveText('Hello, world!');
});

You should immediately see you tests failing with this error:
Expected { selector : 'h1', context : HTMLNode, length : 0 } to have text 'Hello, world!'.

The length : 0 part means that no elements matched the ‘h1′ selector: there aren’t any H1 elements on the page.

To fix this, we’ll need to create an HTML fixture. HTML fixtures can be passed to the setupComponent method call in your test suite (line 7 in hello-world.spec.js):
setupComponent('<div><h1></h1></div>');

The HTML provided will replace the default DIV created by describeComponent.

Your tests should now be failing with this error:
Expected '<h1></h1>' to have text 'Hello, world!'.

This means it can find the H1 tag, but it doesn’t have the correct content.

Hello, world!

Right. It’s time to make our component do what it is supposed to. Open up /app/js/components/hello-world.js.

This is a component definition. We’ll look at how it works in more detail in the next lesson, but for now let’s just modify the method in after(‘initialize’, …) on line 26. This method will be executed when a new instance of the component is created.

this.after('initialize', function () {
this.$node.find('h1').text('Hello, world!');
});

And your tests should now pass.

this.$node is a jQuery object containing a reference to the HTML element to which the component instance is attached. In this case, it will be the DIV from our HTML fixture.

Adding components to your application

Great, we’ve got a working component. Now, go and reload your application, localhost:8080.

Nothing’s changed because we haven’t stated in our application that we want to use the component. Open /app/js/page/main.js. This is a page module (not a component) and will be responsible for loading the components required to run your application.

Line 9 and 22 are commented out examples of how to load and instantiate components:

9 // var myComponent = require('component/my_component');
22 // myComponent.attachTo(document);

Uncomment these lines and modify them to load your hello-world component:

9 var helloWorld = require('component/hello-world');
22 helloWorld.attachTo(document);

You’ll also need to replace the “Hello world” in /app/index.html with an empty H1 tag:

14 <h1></h1>

Now reload your application and gaze upon the wonder of it!

Part 2

In Part 2 we take a step back from the code. We’ll define some user stories, select one to work on and define an event flow to implement.

Making Use of Custom Timelines

Custom Timelines are a new feature for Twitter, allowing you to create a timeline containing specific Tweets. As with all Timelines, it is possible to embed Custom Timelines in a web page, or link to the Timeline directly on Twitter.com.

See the TweetDeck blog post on how to create Custom Timelines.

So, what are they good for? Well…

Live news

Add Tweets to a Custom Timeline to curate an evolving news story.

Stories

Custom Timelines allow you to add Tweets to a Timeline in a specific order. This means you can create a story by selecting Tweets from a variety of sources.

Here’s @passy’s take on Custom Timelines

Q&A

It can be quite hard to follow a Q&A session, as these normally rely on hashtags and can become polluted by spam, and it may be hard to tell which replies are regarding which questions. Custom Timelines allow a user to see the full Q&A without all the noise.

Also, as Custom Timelines are fixed over time, it is possible to preserve the results of a Q&A for all time.

Here’s a @GuardianUS Q&A session on the NSA

Reading List

When you see a Tweet referring to an article, you can now add that Tweet to a Custom Timeline, creating a “to-read” list for your later perusal.

Here’s my growing to-read list (mostly UK news) Postliminary Perusal

Specific Subject

Interested in JavaScript, politics or green energy? You can create a Custom Timeline for that subject, picking the best tweets for sharing with other users. A bit like Tumblr, perhaps?

Here’s the Guardian again with New York travel tips

Media

You can create a Custom Timeline containing your favorite images & Vines. Everyone likes pictures :)

Getting Started with Flight

I’ve written a book! It’s a getting-started guide to Twitter’s JavaScript framework, Flight. If you’re creating JavaScript applications of any sort, I’d really recommend taking a look. There’s lots of lessons-learned in there from my experience working on TweetDeck, as well as basic how-tos for constructing components & mixins, all presented in what I hope is a clear and helpful manner.

It’s available now on Amazon:

0957OS_cov

What is a Flight component?

Twitter’s Flight framework is component-based. So, what is a component?

A component is an atomic piece of functionality. It is atomic because it is self-contained: it does not depend on other components.

A component can be thought of as comprising three distinct parts:

  • Component definition
  • Component factory
  • Component instances

Component definition

A named function which defines the component’s behavior. This includes event listeners & triggers, default configuration and private methods. A component does not have any public methods or members – all interactions are managed through events.

E.g.:

function heartbeat () {
  this.defaultAttrs({
    bpm: 80
  });
  this.beat = function () {
    this.trigger('heartbeat'); // trigger heartbeat event
  };
  this.after('initialize', function () {
    setInterval(this.beat.bind(this), 60000/bpm); 
  });
}

Component factory

Flight’s defineComponent method accepts a component definition and returns a component factory.

  • A component factory is used to create component instances and attach them to the DOM.
  • A component factory can create many component instances, but only one instance may be attached to any one node.
  • A component’s default attributes can be overridden at instantiation

E.g.

HeartBeat = defineComponent(heartbeat);
HeartBeat.attachTo('body');
HeartBeat.attachTo('#child', {
  bpm: 90
});

Component instance

  • A component instance is created using a component factory’s attachTo method.
  • attachTo does not return a reference to the component instance.
  • A component instance can interact with the DOM node it is attached to, and its subtree.
  • A component instance can interact with the rest of the application via events

E.g.

function heartbeatMonitor () {
  this.handleHeartbeat = function () {
    console.log('Still alive.');
  };
  this.after('initialize', function () {
    this.on('heartbeat', this.handleHeartbeat);
  });
}

Test the interface, not the implementation

There’s a common misconception when it comes to writing unit tests that how something works is important. It is not.

Imagine you are testing a simple amplifier. An amplifier’s interface has two inputs and one output: audio in, the volume knob, audio out.

To test whether the amplifier works correctly, I could disassemble it and test every internal component, making sure that all the named parts are present and work as expected. I could check that the colour banding on the resistors is correct. I could run a current through each part of the circuit to check the connections. In fact, while I was building the amplifier, I would do exactly this.

However, to test whether the amp works all that needs to be done is to turn the volume knob and check that the volume changes accordingly.

When writing unit tests it is important to understand that an amplifier is a unit.

Before you start writing tests, first define the interface. Then test that and only that.

RIP TweetDeck! Long Live TweetDeck!

The state of TweetDeck has in recent months been rather mis-represented in the press. As a result many people seem to think TweetDeck is being shut down, though this is definitely not the case.

TweetDeck is shutting down some old applications and concentrating instead on the latest web & desktop offerings.

This was first announced in a post on TweetDeck’s blog: An update on TweetDeck.

What’s staying

The following versions of TweetDeck will continue to be available and actively developed:

It looks like this:
Screen Shot 2013-06-14 at 11.35.46 AM

What’s going

  • TweetDeck AIR (v0.38)
  • Android
  • iOS
  • Facebook integration

So, if you were using AIR (the old Yellow bird version), you can replace it with the Chrome, Windows, Mac or web version.

For mobile users the obvious replacement is Twitter’s official client, available for Android and iOS.

I hope that’s cleared things up.

TweetDeck: Link shortening

TweetDeck automatically shortens any urls you Tweet using the t.co service (or bit.ly – see below). Shortened urls are 22 characters long.

Before you send your Tweet, TweetDeck will show you the full URL you typed in. The short link is not created until it is sent.

When TweetDeck displays Tweets it looks up the original url for any t.co url and displays that instead of the t.co so it becomes clear exactly where a link you are thinking of clicking on is going to take you.

You can test that link shortening is working by entering a very long link. E.g.

http://this.link.is.very.very.very.long.and.can.not.possibly.fit.into.a.single.tweet.but.thatis.okay.because.TweetDeck.will.shortenit.automatically.com

If you paste that link in to the compose box, it’ll tell you you’ve got 118 characters left.

bit.ly

Currently TweetDeck only offers one alternative link shortening service, bit.ly.

To switch to bit.ly, go to Settings > Services and select bit.ly from the dropdown. You will then need to enter your bit.ly username and API key. You can get these on the bit.ly advanced settings page. At the bottom of that page, click “Show legacy API key” and copy them across to TweetDeck.

TweetDeck Filters

TweetDeck offers powerful filters to allow you to find the proverbial needle in a haystack.

Presented below are a few solutions to common problems.

Finding a specific tweet by a specific user

Problem:
Someone Tweeted a Tweet you’d like to find, but it was a few weeks ago so it doesn’t turn up in a search and you don’t want to have to read through weeks of tweets to find it.
Solution:
  1. Create a column for tweets by that user
  2. scroll down until you think the tweet should be in the column
  3. filter the column by a keyword you think was in the tweet.

Finding relevant breaking news

Problem:
Something is happening in the world but, when you add a search column, it’s moving far too fast or is full of too much spam and retweets.
Solution:
  1. Open User filters and select “By: verified users”
  2. Open content filters and select “Retweets: Excluded”
  3. Your timeline should now only contain the latest news from trusted sources.