Posts tagged: data-wrangling

These are some rough notes from what I thought was an interesting keynote from Melissa Terras, Director of the UCL Centre for Digital Humanities, at this year's BL Labs Symposium. Melissa has a blog: Adventures in Digital Cultural Heritage and a recommended book: Defining Digital Humanities Melissa started by asserting that reuse of digital cultural heritage data is still rare, and that preservation of such data is problematic. Of the content digitised in the National Lottery Fund's New Opportunities programme around the turn of the Millennium, ~60% of the content digitised then is no longer available now. [continues...]

I attended EDINA's Geoforum 2014, described as: ... a free all day event aimed at lecturers, researchers and support staff who promote and support the use of geo-services at their institution. I came along primarily to learn more about the services that my own organisation, EDINA, provides in this space. These are my notes from the day - I had to dip in and out of parts of the day to deal with other things so my notes are quite selective - my aim is to give a flavour of the day. [continues...]

Last week I attended an event, organised by BL Labs working with the Arts and Humanities Research Council, at the British Library's Centre for Conservation. The event, described as a showcase event for British Library Labs and AHRC Digital Transformations, consisted of a packed series of presentations - I won't describe them all (and I missed Bill Thompson's talk anyway) but will, instead, pull out some snippets which interested me in particular. [continues...]

Introduction - (warning - old-timer indulgence) From the mid-nineties through to the end of 2006 I earned my living as a developer of Web applications, or as someone managing Web application development projects. I like to think I was quite good at it, and I certainly have a lot of experience. I worked with CGI writing in Perl and a little C, moving into ColdFusion and Java (via JServ - anyone remember that? [continues...]

Over the years I've found the ' Semantic Web' to be an interesting though, at times, faintly worrying concept. It has never much impacted on my work directly, despite my having been embroiled in Web development since, well pretty much, Web development began. Of late I've tried to follow the earnest discussions about how the Semantic Web went all wrong because it was hijacked by the AI enthusiasts, and how it is going to be alright now because a more pragmatic paradigm has gained the upper-hand, that of Linked Data. [continues...]

Over on ZDNet, Paul Miller has blogged some thoughts about what he calls the 'Data Cloud'. He points out that in the evolution of the 'cloud computing' paradigm, the: ...emphasis for much of this wider discussion remains firmly rooted in the realm of computation and storage. On many levels it’s about offloading the costs of scaling and maintaining local infrastructure, and ‘data’ doesn’t really enter the conversation at all. Something is ‘stored,’ but it’s a nameless, faceless, shapeless something that merely exists in order to be stored or computed upon. [continues...]

Here's an interesting approach. Bernhard Haslhofer at Media Spaces has developed OAI2LOD Server, a system which harvests metadata with OAI-PMH, processes the records to create a triple store and exposes interfaces to this for linked-data clients, SPARQL clients and web-browsers. According to the web-page: The OAI2LOD Server exposes any OAI-PMH compliant metadata repository according to the Linked Data guidelines. This makes things and media objects accessible via HTTP URIs and query able via the SPARQL protocol. [continues...]

Facebook, Google And Plaxo Join The DataPortability Workgroup. So, parts of the blogosphere are quite excited by the news that Facebook, previously criticised for being a closed system, has agreed to join the DataPortability Workgroup. According to Duncan Riley, the author of this TechCrunch post: The DataPortability Workgroup is actively working to create the ‘DataPortability Reference Design’ to document the best practices for integrating existing open standards and protocols for maximum interoperability (and here’s the key area) to allow users to access their friends and media across all the applications, social networking sites and widgets that implement the design into their systems. [continues...]

Correction: In a comment, Justin Crites points out, correctly, that Simple DB does not offer 'relational' database functionality, in the sense that it is not an RDBMS. While this is true, I think Simple DB clearly offers functionality which many people get, perhaps inappropriately, from a relational database system. Amazon continue to expand their excellent infrastructure web services. Simple DB - relational databases in the cloud. I'm definitely going to have a look at this - clustering/replicating databases for resilience and performance was extraordinarily difficult when I was developing and supporting enterprise systems a couple of years ago. [continues...]

I came across this quote: The coolest thing to do with your data will be thought of by someone else attributed to Rufus Pollack, in a in a post on the Talis blog, Nodalities. Absolutely, couldn't agree more. The point is not so much whether this statement might be true or not, so much as what it does to your thinking and planning if you decide to take it as an article of faith. [continues...]

Just came across A Periodic Table of Visualization Methods - not all the 'elements' have examples, but I like the idea.

I had a little problem to solve the other day where I wanted to work up a database from scratch, and then transfer data from an existing database into it. On the spur of the moment I decided to see how quickly/easily this could be done in Ruby. Long story short, the thing I learned (obvious in hindsight) is that: require 'rubygems' require_gem 'activerecord' is all you need to get access to all that ActiveRecord modelling goodness outside of the Rails framework proper (assuming Rails is installed on your system). [continues...]

"...a more flexible open source licensing program for the MySQL client libraries, which are used to access the MySQL database from within applications. Previously usable under either a commercial or a GPL open source license, MySQL has developed its Free and Open Source Software (FOSS) Exception policy enabling the libraries to be used under other open source licensing formats, such as Apache and Artistic." Infoworld This is an interesting development for MySQL, which now has both commercial and free offerings. [continues...]