Short URL implementation with WebAPI
Simple URL shortening web service implementd with WebAPI
Today there are multiple URL shortening solution available online for free. However, there is always need to have some existing funtionalities in the house for various reasons.
This functionality can be easily achieved with different approaches. The following is description of short URL service developed on WebAPI and .NET platform.
The goals of solution
- Create unique key for the given URL
- Check if URL is valid and location is reachable before creating keys and adding to database
- Retrieve and redirect URL for the given key
- Cache keys ti reduce database querying
Most recently updated version of this solution is available on GitHub https://github.com/dejanstojanovic/ShortUrl
Creating unique key for the URL
You can create your unique keys for short URL in different ways. The simplest is to just use number which will be auto-incermented as a primary key in the table. In case you want to do it in bit.ly style you can involve GUIDs. GUIDS are generally larger than just 4 or 6 characters, but that does not mean you can use only one part of GUID.
Of course, since you are not using full GUID you cannot expect the same level of uniqueness as combinations will start repeating after some time. It is nice to have short key, let's say only 4 characters, but that will limit you to not so big number of unique keys.
36×36×36×36=36^4=1679616 distinct strings of length 4
Because of this reason you might want to use 6 characters key which will give you more space for unique keys for short URLs
36×36×36×36x36x36=36^6=2176782336 distinct strings of length 6
In this implementation I decided to use 6 characters configurable implementation, meaning default number of characters for the key is 6 but can be increased or reduced in config. The following is code snippet which checks if there is a key in cache or database already for the URL, if not than it creates it, caches it and returns it.
public class UrlManager
{
private ShortUrl.Data.Context dbContext;
private bool CheckUrlAvailability
{
get
{
bool check;
if(!bool.TryParse(ConfigurationManager.AppSettings["CheckUrlAvailability"], out check))
{
check = false;
}
return check;
}
}
private int CheckUrlAvailabilityTimeout
{
get
{
int timeout;
if (!int.TryParse(ConfigurationManager.AppSettings["CheckUrlAvailabilityTimeout"], out timeout))
{
timeout = 5;
}
return timeout;
}
}
private int KeyLength
{
get
{
int keyLength;
if( !int.TryParse(ConfigurationManager.AppSettings["KeyLength"], out keyLength))
{
keyLength = 6;
}
return keyLength;
}
}
private int CacheTimeout
{
get
{
int cacheTimeout;
if(!int.TryParse(ConfigurationManager.AppSettings["CaheTimeout"], out cacheTimeout))
{
cacheTimeout = 5;
}
return cacheTimeout;
}
}
public UrlManager()
{
dbContext = new Data.Context();
}
public String AddShortUrl(string Url)
{
if (!string.IsNullOrWhiteSpace(Url))
{
Url = Url.Trim().ToLower();
}
String cached = null;
cached = this.GetCached<String>(Url);
if (!String.IsNullOrWhiteSpace(cached))
{
return cached;
}
try
{
var url = new Uri(Url);
}
catch(Exception ex)
{
throw new Exceptions.InvalidUrlException(ex);
}
if ((this.CheckUrlAvailability && this.HttpGetStatusCode(Url).Result == HttpStatusCode.OK) || !this.CheckUrlAvailability)
{
String newKey = null;
while (string.IsNullOrEmpty(newKey))
{
if (!dbContext.ShortUrls.Any(s => s.Url == Url))
{
newKey = Guid.NewGuid().ToString("N").Substring(0, this.KeyLength).ToLower();
dbContext.ShortUrls.Add(new Data.Models.ShortUrl() { Key = newKey, Url = Url, DateCreated = DateTime.Now });
dbContext.SaveChanges();
}
else
{
var shortUrl = dbContext.ShortUrls.Where(s => s.Url == Url).FirstOrDefault();
if (shortUrl != null)
{
newKey = shortUrl.Key;
}
}
}
this.TryAddToCache<String>(Url, newKey);
return newKey;
}
else
{
throw new Exceptions.UrlUnreachableException(null);
}
}
public String GetUrl(String ShortUrlKey)
{
var url = dbContext.ShortUrls.Where(s => s.Key == ShortUrlKey).FirstOrDefault();
if (url != null)
{
return url.Url;
}
else
{
throw new Exceptions.MissingKeyException(null);
}
}
public async Task<HttpStatusCode> HttpGetStatusCode(string Url)
{
try
{
var httpclient = new HttpClient();
httpclient.Timeout = TimeSpan.FromSeconds(this.CacheTimeout);
var response = await httpclient.GetAsync(Url, HttpCompletionOption.ResponseHeadersRead);
string text = null;
using (var stream = await response.Content.ReadAsStreamAsync())
{
var bytes = new byte[10];
var bytesread = stream.Read(bytes, 0, 10);
stream.Close();
text = Encoding.UTF8.GetString(bytes);
Console.WriteLine(text);
}
return response.StatusCode;
}
catch (Exception ex)
{
return HttpStatusCode.NotFound;
}
}
private T GetCached<T>(string cacheKey) where T : class
{
HttpContext httpContext = HttpContext.Current;
if (httpContext != null)
{
return HttpContext.Current.Cache[cacheKey] as T;
}
return null;
}
private bool TryAddToCache<T>(string cacheKey, T value, int timeout = 0)
{
HttpContext httpContext = HttpContext.Current;
if (httpContext != null)
{
//httpContext.Cache.Add(cacheKey, value, null, DateTime.Now.AddMinutes(cacheTimeout), Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);
httpContext.Cache.Insert(cacheKey, value, null, Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(this.CacheTimeout));
}
return false;
}
}
To test adding of URL ti the database I used Postman. Since it is tricky to send plain text to WebAPI controller action, I did the test with JSON string data since anyway, from the client web UI most likely you would have to post JSON from JQuery of AngularJS
Check if URL is valid and location is reachable before creating keys and adding to database
Since we want to protect our database to store "junk" URLs, we need to check if the URL string is a valid URL. This is easy to implement by trying to create instance of System.Uri class.
Second thing is to check if the URL is actually a reachable location. We can simply make web request to URL, but since we do not need the whole request and we only need the response code I did it a bit different to reduce the download amount and unnecessary traffic. Instead of taking the whole response, we only read first 100 bytes.
public async Task<HttpStatusCode> HttpGetStatusCode(string Url)
{
try
{
var httpclient = new HttpClient();
httpclient.Timeout = TimeSpan.FromSeconds(this.CacheTimeout);
var response = await httpclient.GetAsync(Url, HttpCompletionOption.ResponseHeadersRead);
string text = null;
using (var stream = await response.Content.ReadAsStreamAsync())
{
var bytes = new byte[10];
var bytesread = stream.Read(bytes, 0, 10);
stream.Close();
text = Encoding.UTF8.GetString(bytes);
Console.WriteLine(text);
}
return response.StatusCode;
}
catch (Exception ex)
{
return HttpStatusCode.NotFound;
}
}
This check is optional and it is configurable from the config file.
Current implementation only check for 200 OK status code as a valid status code for adding the URL. In case you want to add URLs with different status codes like login protected codes you can easily extend the condition in UrlManager class
Retrieve and redirect URL for the given key
Now when you have the URL stored with generated key, you need to make redirection when requested. Since the implementation is done with WebAPI, we need to tell the browser to redirect and we can do this easily with response code 301 (System.Net.HttpStatusCode.Moved)
[HttpGet]
[Route("{key}")]
public HttpResponseMessage Get(string key)
{
var response = Request.CreateResponse(HttpStatusCode.Moved);
String urlString = new ShortUrl.Logic.UrlManager().GetUrl(key);
if (!String.IsNullOrWhiteSpace(urlString))
{
response.Headers.Location = new Uri(urlString);
return response;
}
return null;
}
The routing rule for Get action does not include any prefix so for getting the redirection for the short URL key you only need to invoke the domain of your service with the short key. For our case in the development environment this would be in the following form http://localhost:37626/993ef8 the same way bit.ly and other URL shortening services.
Cache keys ti reduce database querying
Since every get request will hit your controller action for redirection, you might get your database server pretty busy since every request will got ot database and back. To avoid this you can use caching.
In this implementation I used System.Web.Caching which limits the solution to IIS hosting. In case you do not host your service inside on IIS this caching will not work
The caching keeps keys and URLs for certain amount of minutes and resets the cache timeout every time value is used. This means that values used more frequently will be cached for longer time, which is good. The items rarely used will expire from cache and that way it will maintain same amount of cache memory used.
Since cache collection is thread safe by default, so no need additional collection locking while reading and writing to it.
private T GetCached<T>(string cacheKey) where T : class
{
HttpContext httpContext = HttpContext.Current;
if (httpContext != null)
{
return HttpContext.Current.Cache[cacheKey] as T;
}
return null;
}
private bool TryAddToCache<T>(string cacheKey, T value, int timeout = 0)
{
HttpContext httpContext = HttpContext.Current;
if (httpContext != null)
{
httpContext.Cache.Insert(cacheKey, value, null, Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(this.CacheTimeout));
}
return false;
}
All the basic functionalities are configurable, and there is a specific value in web.config which determines behavior of
- Key generating process
- Checking the location availability
- Caching
- Check if the URL is a reachable location
<appSettings>
<add key="CheckUrlAvailability" value="True"/>
<add key="CheckUrlAvailabilityTimeout" value="5"/>
<add key="KeyLength" value="6"/>
<add key="CacheTimeout" value="5"/>
</appSettings>
Initial version of the solution for this implementatio is available for download from this article or you can fork most recently uppdated version on GitHub https://github.com/dejanstojanovic/ShortUrl.
References
- http://www.asp.net/web-api
- https://msdn.microsoft.com/en-us/library/hh833994(v=vs.108).aspx
- https://github.com/dejanstojanovic/ShortUrl
Disclaimer
Purpose of the code contained in snippets or available for download in this article is solely for learning and demo purposes. Author will not be held responsible for any failure or damages caused due to any other usage.
About the author
DEJAN STOJANOVICDejan is a passionate Software Architect/Developer. He is highly experienced in .NET programming platform including ASP.NET MVC and WebApi. He likes working on new technologies and exciting challenging projects
Comments for this article