Skip to main content

Posts

Showing posts with the label dataset

Diamond installation on centos 7

$  yum install make rpm-build python-configobj python-setuptools $  git clone  https://github.com/python-diamond/Diamond $  cd Diamond $  make buildrpm Then use the package you built like this: $  yum localinstall –nogpgcheck dist/diamond-4.0.449-0.noarch.rpm $  cp /etc/diamond/{diamond.conf.example,diamond.conf} $  $EDITOR /etc/diamond/diamond.conf # Start Diamond service via service manager. $  service diamond start diamond-setup -C ElasticSearchCollector diamond-setup -C NetworkCollector Issues failed to connect socket to ‘/var/run/libvirt/libvirt-sock-ro’ no such file or directory execute: egrep ‘(vmx|svm)’ /proc/cpuinfo 2. If the above commands returns with any output showing vmx or svm then your hardware supports VT else it does not. yum install qemu-kvm qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer

Diamond installation on centos 7

$ yum install make rpm-build python-configobj python-setuptools $ git clone https://github.com/python-diamond/Diamond $ cd Diamond $ make buildrpm Then use the package you built like this: $ yum localinstall --nogpgcheck dist/diamond-4.0.449-0.noarch.rpm $ cp /etc/diamond/{diamond.conf.example,diamond.conf} $ $EDITOR /etc/diamond/diamond.conf # Start Diamond service via service manager. $ service diamond start diamond-setup -C ElasticSearchCollector diamond-setup -C NetworkCollector Issues failed to connect socket to '/var/run/libvirt/libvirt-sock-ro' no such file or directory execute: egrep '(vmx|svm)' /proc/cpuinfo 2. If the above commands returns with any output showing vmx or svm then your hardware supports VT else it does not. yum install qemu-kvm qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer

The Ceres Database

Ceres is a time-series database format intended to replace Whisper as the default storage format for Graphite. In contrast with Whisper, Ceres is not a fixed-size database and is designed to better support sparse data of arbitrary fixed-size resolutions. This allows Graphite to distribute individual time-series across multiple servers or mounts. Ceres is not actively developped at the moment. For alternatives to whisper look at  alternative storage backends . Storage Overview Ceres databases are comprised of a single tree contained within a single path on disk that stores all metrics in nesting directories as nodes. A Ceres node represents a single time-series metric, and is composed of at least two data files. A slice to store all data points, and an arbitrary key-value metadata file. The minimum required metadata a node needs is a  'timeStep' . This setting is the finest resolution that can be used for writing. A Ceres node however can contain and read data with other...

The Ceres Database

Ceres is a time-series database format intended to replace Whisper as the default storage format for Graphite. In contrast with Whisper, Ceres is not a fixed-size database and is designed to better support sparse data of arbitrary fixed-size resolutions. This allows Graphite to distribute individual time-series across multiple servers or mounts. Ceres is not actively developped at the moment. For alternatives to whisper look at  alternative storage backends . Storage Overview Ceres databases are comprised of a single tree contained within a single path on disk that stores all metrics in nesting directories as nodes. A Ceres node represents a single time-series metric, and is composed of at least two data files. A slice to store all data points, and an arbitrary key-value metadata file. The minimum required metadata a node needs is a  'timeStep' . This setting is the finest resolution that can be used for writing. A Ceres node however can contain and read data with other, less-p...

Graphite and your data

Getting your data into Graphite is very flexible. There are three main methods for sending data to Graphite: Plaintext, Pickle, and AMQP. It’s worth noting that data sent to Graphite is actually sent to the  Carbon and Carbon-Relay , which then manage the data. The Graphite web interface reads this data back out, either from cache or straight off disk. Choosing the right transfer method for you is dependent on how you want to build your application or script to send data: There are some tools and APIs which can help you get your data into Carbon. For a singular script, or for test data, the plaintext protocol is the most straightforward method. For sending large amounts of data, you’ll want to batch this data up and send it to Carbon’s pickle receiver. Finally, Carbon can listen to a message bus, via AMQP. The plaintext protocol The plaintext protocol is the most straightforward protocol supported by Carbon. The data sent must be in the following format:  <me...

Graphite and your data

Getting your data into Graphite is very flexible. There are three main methods for sending data to Graphite: Plaintext, Pickle, and AMQP. It’s worth noting that data sent to Graphite is actually sent to the  Carbon and Carbon-Relay , which then manage the data. The Graphite web interface reads this data back out, either from cache or straight off disk. Choosing the right transfer method for you is dependent on how you want to build your application or script to send data: There are some tools and APIs which can help you get your data into Carbon. For a singular script, or for test data, the plaintext protocol is the most straightforward method. For sending large amounts of data, you’ll want to batch this data up and send it to Carbon’s pickle receiver. Finally, Carbon can listen to a message bus, via AMQP. The plaintext protocol The plaintext protocol is the most straightforward protocol supported by Carbon. The data sent must be in the following format:  <metric   path> ...

How Machines Make Sense of Big Data: an Introduction to Clustering Algorithms

Take a look at the image below. It’s a collection of bugs and creepy-crawlies of different shapes and sizes. Take a moment to categorize them by similarity into a number of groups. This isn’t a trick question. Start with grouping the spiders together.   Images via Google Image Search, labelled for reuse Done? While there’s not necessarily a “correct” answer here, it’s most likely you split the bugs into four  clusters . The spiders in one cluster, the pair of snails in another, the butterflies and moth into one, and the trio of wasps and bees into one more. That wasn’t too bad, was it? You could probably do the same with twice as many bugs, right? If you had a bit of time to spare — or a passion for entomology — you could probably even do the same with a hundred bugs. For a machine though, grouping ten objects into however many meaningful clusters is no small task, thanks to a mind-bending branch of maths called  combinatorics , which tells us th...