Data Monitor - Revision history

70.174.157.161: /* Juan's work */

2010-07-31T06:37:48Z

Juan's work

← Older revision		Revision as of 02:37, 31 July 2010
Line 28:		Line 28:


	At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity. Any group that changes by 50% or more, with respect to the reference, is flagged as only a warning. If greater than 80% however, they are marked as errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.		At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity. Any group that changes by 60% or more, with respect to the reference, is flagged as only a warning. If greater than 80% however, they are marked as errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.


	Alarms are stored in arrays known as warnlist and errorlist. These lists are then scanned to assure that no hardware alarms overlap. For example, if an entire ADB crate is malfunctioning, all associated ADB board warnings will be cleared. This sorting prevents redundancy when sending alarms. No implementation is in place for overlapping, non-associative units. For example, if a TDC crate malfunctions, all fuses corresponding to that section will also be flagged. If greater than 200 alarms are triggered then a single message informs the user of a universal drop rather than bombarding them with error messages. Below is a list of how the algorithm prioritizes alarms with respect to corresponding units.		Alarms are stored in arrays known as warnlist and errorlist. These lists are then scanned to assure that no hardware alarms overlap. For example, if an entire ADB crate is malfunctioning, all associated ADB board warnings will be cleared. This sorting prevents redundancy when sending alarms. No implementation is in place for overlapping, non-associative units(besides LV supply and ADB crates). For example, if a TDC crate malfunctions, all fuses corresponding to that section will also be flagged. If greater than 200 alarms are triggered then a single message informs the user of a universal drop rather than bombarding them with error messages. Below is a list of how the algorithm prioritizes alarms with respect to corresponding units.


Line 41:		Line 41:

	*LV supply > fuses		*LV supply > fuses


			The algorithm used for overlapping LV supplies and ADB crates is based on the layers associated with both. Upon finding alarming activity, the algorithm checks the malfunctioning hardware's config file and stores some key variables associated with it. The stored variables are:


			*sector
			*highest and lowest layer number


			If a LV supply malfunction is found, the algorithm scans the entire alarm list to check for ADB crates. Using the variables collected from the config file, (those listed above) it checks to see if the LV supply and ADB crate are in the same sector and whether the range of ADB crate layers are within the LV supply's. Here is a hypothetical example of an ADB crate that would be removed:

			*LV supply sector:6 low:10 high:24
			*ADB crate sector:6 low:17 high:22 <---the ADB crate's range is 17-22 while the LV supply's is 10-24. Since it falls in inside LV supply's range, the ADB crate should be removed to prevent redundancy in alarms

98.166.36.226: /* Juan's work */

2010-07-22T21:14:55Z

Juan's work

← Older revision		Revision as of 17:14, 22 July 2010
Line 35:		Line 35:

	Corresponding Units:		Corresponding Units:
	-TDC crate > TDC boards
	-ADB crate > ADB boards		*TDC crate > TDC boards
	-LV supply > fuses
			*ADB crate > ADB boards

			*LV supply > fuses

98.166.36.226: /* Juan's work */

2010-07-22T20:08:13Z

Juan's work

← Older revision		Revision as of 16:08, 22 July 2010
Line 28:		Line 28:


	At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity ~~occurs~~. Any group that changes by 50% or more, with respect to the reference, is ~~marked~~ as a warning. If greater than 80% however, they are marked as ~~true~~ errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.		At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity. Any group that changes by 50% or more, with respect to the reference, is flagged as only a warning. If greater than 80% however, they are marked as errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.

98.166.36.226: /* Juan's work */

2010-07-22T20:05:25Z

Juan's work

← Older revision		Revision as of 16:05, 22 July 2010
Line 26:		Line 26:

	The file dmlibtest.c analyzes data from drift chamber wires to determine whether there exists a hardware malfunction within the drift chamber. After 90,000 events (~30 seconds) the program stores the hits from each wire into groups designated by corresponding hardware (ex:fuse, ADB boards/crates). Using these groups as a reference, the program can look at future events to scan for alarming activity.		The file dmlibtest.c analyzes data from drift chamber wires to determine whether there exists a hardware malfunction within the drift chamber. After 90,000 events (~30 seconds) the program stores the hits from each wire into groups designated by corresponding hardware (ex:fuse, ADB boards/crates). Using these groups as a reference, the program can look at future events to scan for alarming activity.


	At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity occurs. Any group that changes by 50% or more, with respect to the reference, is marked as a warning. If greater than 80% however, they are marked as true errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.		At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity occurs. Any group that changes by 50% or more, with respect to the reference, is marked as a warning. If greater than 80% however, they are marked as true errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.


	Alarms are stored in arrays known as warnlist and errorlist. These lists are then scanned to assure that no hardware alarms overlap. For example, if an entire ADB crate is malfunctioning, all associated ADB board warnings will be cleared. This sorting prevents redundancy when sending alarms. If greater than 200 alarms are triggered then a single message informs the user of a universal drop rather than bombarding them with error messages.		Alarms are stored in arrays known as warnlist and errorlist. These lists are then scanned to assure that no hardware alarms overlap. For example, if an entire ADB crate is malfunctioning, all associated ADB board warnings will be cleared. This sorting prevents redundancy when sending alarms. No implementation is in place for overlapping, non-associative units. For example, if a TDC crate malfunctions, all fuses corresponding to that section will also be flagged. If greater than 200 alarms are triggered then a single message informs the user of a universal drop rather than bombarding them with error messages. Below is a list of how the algorithm prioritizes alarms with respect to corresponding units.


			Corresponding Units:
			-TDC crate > TDC boards
			-ADB crate > ADB boards
			-LV supply > fuses


	Another file, juanlib.c, creates histogram files from the data collected in a similar fashion. Marked as 101-106, 201-206 and 301-306, these plots correspond to data from the 6 respective sectors. Histograms 101-106 correspond to one dimensional histograms, 201-206 to two dimensional histograms (hits in wire and layer), and 301-306 to normalized changes in 6x6 wire groups with respect to the reference. The box feature is recommended for the 301-306 plots.		Another file, juanlib.c, creates histogram files from the data collected in a similar fashion. Marked as 101-106, 201-206 and 301-306, these plots correspond to data from the 6 respective sectors. Histograms 101-106 correspond to one dimensional histograms, 201-206 to two dimensional histograms (hits in wire and layer), and 301-306 to normalized changes in 6x6 wire groups with respect to the reference. The box feature is recommended for the 301-306 plots.

98.166.36.226: /* Juan's work */

2010-07-22T18:33:35Z

Juan's work

← Older revision		Revision as of 14:33, 22 July 2010
Line 25:		Line 25:
	== Juan's work ==		== Juan's work ==

	The file ~~juanlib~~.c analyzes data from drift chamber wires to determine whether there exists a hardware malfunction within the drift chamber. After 90,000 events (~30 seconds) the program stores the hits from each wire into ~~6x6~~ groups designated by ~~sector~~, ~~superlayer, and wire number~~. Using these groups as a reference, the program can look at future events to scan for alarming activity.		The file dmlibtest.c analyzes data from drift chamber wires to determine whether there exists a hardware malfunction within the drift chamber. After 90,000 events (~30 seconds) the program stores the hits from each wire into groups designated by corresponding hardware (ex:fuse, ADB boards/crates). Using these groups as a reference, the program can look at future events to scan for alarming activity.

			At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity occurs. Any group that changes by 50% or more, with respect to the reference, is marked as a warning. If greater than 80% however, they are marked as true errors. Thresholds were determined by trial and error on many data sets. There exists a phenomenon in which a drop of roughly 60% occurs across all regions occurs. This has not yet been dealt with by the algorithm.

	At the 180,000 event mark and for every 90,000 events afterward, the program compares the past 90,000 events with the reference to determine whether suspicious activity occurs. Any 6x6 group that changes by 60% or more, with respect to the reference, is marked as an error. The 60% threshold was determined by trial and error on many data sets. A drop of roughly 60% across all regions generally corresponds to a drop in beam current. Errors are stored in a separate data structure named errorlist.

			Alarms are stored in arrays known as warnlist and errorlist. These lists are then scanned to assure that no hardware alarms overlap. For example, if an entire ADB crate is malfunctioning, all associated ADB board warnings will be cleared. This sorting prevents redundancy when sending alarms. If greater than 200 alarms are triggered then a single message informs the user of a universal drop rather than bombarding them with error messages.

	Alarms are only sent if greater than 5 errors occur and only if they occur within the same sector. This was implemented due to small inconsistencies with beam current leading to small regions being flagged as errors. If 5 or more of these errors fall within the same superlayer only a superlayer alarm will occur, ~~rather than multiple 6x6 alarms~~. ~~If greater than 100 errors are recorded~~, ~~only an alarm warning that major drops are found in many areas will appear. This type of alarm usually corresponds to a drop in beam current.~~		Another file, juanlib.c, creates histogram files from the data collected in a similar fashion. Marked as 101-106, 201-206 and 301-306, these plots correspond to data from the 6 respective sectors. Histograms 101-106 correspond to one dimensional histograms, 201-206 to two dimensional histograms (hits in wire and layer), and 301-306 to normalized changes in 6x6 wire groups with respect to the reference. The box feature is recommended for the 301-306 plots.


	~~Juanlib.c also~~ creates histogram files from the data collected. Marked as 101-106, 201-206 and 301-306, these plots correspond to data from the 6 respective sectors. Histograms 101-106 correspond to one dimensional histograms, 201-206 to two dimensional histograms (hits in wire and layer), and 301-306 to normalized changes in 6x6 groups with respect to the reference. The box feature is recommended for the 301-306 plots.

	== Drift Chamber Hardware Mapping ==		== Drift Chamber Hardware Mapping ==

98.166.36.226: /* to do */

2010-07-22T18:03:11Z

to do

← Older revision		Revision as of 14:03, 22 July 2010
Line 6:		Line 6:

	1. Normalization:		1. Normalization:

	* on reference (current algorithm)

	* on the naighbors of the same kind

	* on the same object in another sector(s), probably averaging 6 or 5 sectors, or averaging 3 sectors with medium counts etc

	* if running without beam (cosmic) for long time, should recognize it and give ONE warning for entire DC		* if running without beam (cosmic) for long time, should recognize it and give ONE warning for entire DC
Line 19:		Line 13:

	* warning messages (hot channels (50% up, 120% down etc); if up more then 300% up, discuss with Mac		* warning messages (hot channels (50% up, 120% down etc); if up more then 300% up, discuss with Mac

	* error messages: sharp drop (almost to zero, maybe 90%)

	* any unusual effects - discuss with Mac		* any unusual effects - discuss with Mac

129.57.76.146: /* to do */

2010-07-20T14:45:38Z

to do

← Older revision		Revision as of 10:45, 20 July 2010
Line 3:		Line 3:
	== to do ==		== to do ==

	Sergey (Moscow):		Sergey: Moscow cell phone: 9-011-7-916-729-8203, skype: boiarino

	1. Normalization:		1. Normalization:
Line 30:		Line 30:

	* wiki manual with algorithm description		* wiki manual with algorithm description




	== Juan's work ==		== Juan's work ==

129.57.76.146 at 14:39, 20 July 2010

2010-07-20T14:39:58Z

← Older revision		Revision as of 10:39, 20 July 2010
Line 30:		Line 30:

	* wiki manual with algorithm description		* wiki manual with algorithm description




	== Juan's work ==		== Juan's work ==

129.57.76.146: /* to do */

2010-07-20T14:39:40Z

to do

← Older revision		Revision as of 10:39, 20 July 2010
Line 2:		Line 2:

	== to do ==		== to do ==

			Sergey (Moscow):

	1. Normalization:		1. Normalization:
Line 10:		Line 12:

	* on the same object in another sector(s), probably averaging 6 or 5 sectors, or averaging 3 sectors with medium counts etc		* on the same object in another sector(s), probably averaging 6 or 5 sectors, or averaging 3 sectors with medium counts etc

			* if running without beam (cosmic) for long time, should recognize it and give ONE warning for entire DC


	2. Reporting:		2. Reporting:

	* warning messages (hot channels (50% up, 120% down etc)		* warning messages (hot channels (50% up, 120% down etc); if up more then 300% up, discuss with Mac

			* error messages: sharp drop (almost to zero, maybe 90%)

			* any unusual effects - discuss with Mac


			3. Results:

			* program

			* wiki manual with algorithm description

	== Juan's work ==		== Juan's work ==

129.57.76.146 at 14:14, 20 July 2010

2010-07-20T14:14:43Z

← Older revision		Revision as of 10:14, 20 July 2010
Line 1:		Line 1:
	'''Data Monitor''' is the data quality check program. It runs online being attached to the [[ET System]].		'''Data Monitor''' is the data quality check program. It runs online being attached to the [[ET System]].

			== to do ==

			1. Normalization:

			* on reference (current algorithm)

			* on the naighbors of the same kind

			* on the same object in another sector(s), probably averaging 6 or 5 sectors, or averaging 3 sectors with medium counts etc

			2. Reporting:

			* warning messages (hot channels (50% up, 120% down etc)


	== Juan's work ==		== Juan's work ==